[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029620#comment-17029620
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #262 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/262/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/262//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/262//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/262//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22749) Distributed MOB compactions
[
https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029591#comment-17029591
]
HBase QA commented on HBASE-22749:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
31s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 23 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
1s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:orange}-0{color} | {color:orange} patch {color} | {color:orange} 4m
18s{color} | {color:orange} Used diff version of patch file. Binary files and
potentially other changes not applied. Please rebase and squash commits if
necessary. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 13 new + 432
unchanged - 53 fixed = 445 total (was 485) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} hbase-it: The patch generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 4 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
41s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
27s{color} | {color:red} hbase-server generated 1 new + 0 unchanged - 0 fixed =
1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}151m 45s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[GitHub] [hbase] Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions
Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-581757042 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 23 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 14s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | checkstyle | 1m 37s | master passed | | +1 :green_heart: | shadedjars | 4m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 53s | master passed | | +0 :ok: | spotbugs | 4m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 0s | master passed | | -0 :warning: | patch | 4m 18s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | -1 :x: | checkstyle | 1m 18s | hbase-server: The patch generated 13 new + 432 unchanged - 53 fixed = 445 total (was 485) | | -1 :x: | checkstyle | 0m 17s | hbase-it: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 :x: | whitespace | 0m 0s | The patch 4 line(s) with tabs. | | +1 :green_heart: | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 51s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | | -1 :x: | findbugs | 4m 27s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | -1 :x: | unit | 151m 45s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 1m 12s | hbase-it in the patch passed. | | +1 :green_heart: | asflicense | 1m 8s | The patch does not generate ASF License warnings. | | | | 214m 6s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Useless condition:succeed == true at this point At MobFileCleanerChore.java:[line 250] | | Failed junit tests | hadoop.hbase.mob.TestMobCompactionRegularMode | | | hadoop.hbase.mob.TestMobCompactionRegularRegionBatchMode | | | hadoop.hbase.mob.TestMobCompactionOptRegionBatchMode | | | hadoop.hbase.mob.TestMobCompactionOptMode | | | hadoop.hbase.mob.TestMobFileCleanerChore | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/921 | | JIRA Issue | HBASE-22749 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a52f7813cb63 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-921/out/precommit/personality/provided.sh | | git revision | master / d797ed04e5 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/diff-checkstyle-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/diff-checkstyle-hbase-it.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/whitespace-tabs.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/2/artifact/out/patch-unit-hbase-server.txt | | Test Results |
[jira] [Created] (HBASE-23789) [Flakey Tests] ERROR [Time-limited test] balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file located at ' /tmp/hbase-balancer.rules '
Michael Stack created HBASE-23789:
-
Summary: [Flakey Tests] ERROR [Time-limited test]
balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file
located at ' /tmp/hbase-balancer.rules '
Key: HBASE-23789
URL: https://issues.apache.org/jira/browse/HBASE-23789
Project: HBase
Issue Type: Bug
Components: flakies
Reporter: Michael Stack
We can't find the balancer rules we just read in the
HeterogeneousRegionCountCostFunction test in high load conditions
{code}
2020-02-03 20:51:00,774 ERROR [Time-limited test]
balancer.HeterogeneousRegionCountCostFunction(199): cannot read rules file
located at ' /tmp/hbase-balancer.rules ':File /tmp/hbase-balancer.rules does
not exist
2020-02-03 20:51:00,774 WARN [Time-limited test]
balancer.HeterogeneousRegionCountCostFunction(155): cannot load rules file,
keeping latest rules file which has 1 rules
{code}
Test then goes on to fail with:
{code}
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testOneGroup
Time elapsed: 15.223 s <<< FAILURE!
junit.framework.AssertionFailedError: Host rs0 should be below 0.0%
at
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testWithCluster(TestStochasticLoadBalancerHeterogeneousCost.java:209)
at
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testHeterogeneousWithCluster(TestStochasticLoadBalancerHeterogeneousCost.java:160)
at
org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerHeterogeneousCost.testOneGroup(TestStochasticLoadBalancerHeterogeneousCost.java:102)
{code}
Instead, have tests write rules to local test dir.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log
virajjasani commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-581747580 @saintstack I will start creating necessary sub-tasks to this. For now, having ring buffer might be a good addition to 2.3.0 release for debugging slow RPC calls. Please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23350) Make compaction files cacheonWrite configurable based on threshold
[ https://issues.apache.org/jira/browse/HBASE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029581#comment-17029581 ] Abhinaba Sarkar commented on HBASE-23350: - Already merged. Builds are fine. Resolving the issue > Make compaction files cacheonWrite configurable based on threshold > -- > > Key: HBASE-23350 > URL: https://issues.apache.org/jira/browse/HBASE-23350 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: ramkrishna.s.vasudevan >Assignee: Abhinaba Sarkar >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > As per comment from [~javaman_chen] in the parent JIRA > https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16937361=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937361 > This is to introduce a config to identify if the resulting compacted file's > blocks should be added to the cache - while writing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23788) ROW_INDEX_V1 encoder should consider the secondary index size with the encoded data size tracking
[ https://issues.apache.org/jira/browse/HBASE-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-23788: --- Description: Copied from parent issue comments "In case of Row Index this tracking is not accounting the row offsets index being written. Because on the go these are kept in memory only and written at the end of block write. This end block decision was based on size check only which did NOT include this offsets index." > ROW_INDEX_V1 encoder should consider the secondary index size with the > encoded data size tracking > - > > Key: HBASE-23788 > URL: https://issues.apache.org/jira/browse/HBASE-23788 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Priority: Major > > Copied from parent issue comments > "In case of Row Index this tracking is not accounting the row offsets index > being written. Because on the go these are kept in memory only and written at > the end of block write. This end block decision was based on size check only > which did NOT include this offsets index." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HBASE-23788) ROW_INDEX_V1 encoder should consider the secondary index size with the encoded data size tracking
[ https://issues.apache.org/jira/browse/HBASE-23788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John reassigned HBASE-23788: -- Assignee: Abhinaba Sarkar > ROW_INDEX_V1 encoder should consider the secondary index size with the > encoded data size tracking > - > > Key: HBASE-23788 > URL: https://issues.apache.org/jira/browse/HBASE-23788 > Project: HBase > Issue Type: Sub-task >Reporter: Anoop Sam John >Assignee: Abhinaba Sarkar >Priority: Major > > Copied from parent issue comments > "In case of Row Index this tracking is not accounting the row offsets index > being written. Because on the go these are kept in memory only and written at > the end of block write. This end block decision was based on size check only > which did NOT include this offsets index." -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23788) ROW_INDEX_V1 encoder should consider the secondary index size with the encoded data size tracking
Anoop Sam John created HBASE-23788: -- Summary: ROW_INDEX_V1 encoder should consider the secondary index size with the encoded data size tracking Key: HBASE-23788 URL: https://issues.apache.org/jira/browse/HBASE-23788 Project: HBase Issue Type: Sub-task Reporter: Anoop Sam John -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23279) Switch default block encoding to ROW_INDEX_V1
[ https://issues.apache.org/jira/browse/HBASE-23279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-23279: --- Issue Type: Improvement (was: Wish) > Switch default block encoding to ROW_INDEX_V1 > - > > Key: HBASE-23279 > URL: https://issues.apache.org/jira/browse/HBASE-23279 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0, 2.3.0 >Reporter: Lars Hofhansl >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23279.master.000.patch, > HBASE-23279.master.001.patch, HBASE-23279.master.002.patch, > HBASE-23279.master.003.patch, HBASE-23279.master.004.patch, > HBASE-23279.master.005.patch > > > Currently we set both block encoding and compression to NONE. > ROW_INDEX_V1 has many advantages and (almost) no disadvantages (the hfiles > are slightly larger about 3% or so). I think that would a better default than > NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] abhinabasarkar closed pull request #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold
abhinabasarkar closed pull request #1074: HBASE-23350 Make compaction files cacheonWrite configurable based on threshold URL: https://github.com/apache/hbase/pull/1074 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23350) Make compaction files cacheonWrite configurable based on threshold
[ https://issues.apache.org/jira/browse/HBASE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhinaba Sarkar updated HBASE-23350: Resolution: Fixed Status: Resolved (was: Patch Available) > Make compaction files cacheonWrite configurable based on threshold > -- > > Key: HBASE-23350 > URL: https://issues.apache.org/jira/browse/HBASE-23350 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: ramkrishna.s.vasudevan >Assignee: Abhinaba Sarkar >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > As per comment from [~javaman_chen] in the parent JIRA > https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16937361=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937361 > This is to introduce a config to identify if the resulting compacted file's > blocks should be added to the cache - while writing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119#issuecomment-581733371 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 19 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 47s | branch-2 passed | | +1 :green_heart: | compile | 2m 23s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 17s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 26s | branch-2 passed | | +0 :ok: | spotbugs | 0m 34s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 8m 11s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 21s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | cc | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | -1 :x: | checkstyle | 0m 37s | hbase-client: The patch generated 1 new + 194 unchanged - 0 fixed = 195 total (was 194) | | -1 :x: | checkstyle | 1m 14s | hbase-server: The patch generated 13 new + 119 unchanged - 0 fixed = 132 total (was 119) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 4s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 2m 12s | the patch passed | | +1 :green_heart: | javadoc | 1m 27s | the patch passed | | +1 :green_heart: | findbugs | 8m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 42s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 3m 32s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 48s | hbase-zookeeper in the patch passed. | | +1 :green_heart: | unit | 155m 2s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 32s | The patch does not generate ASF License warnings. | | | | 239m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1119 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux b1f1ac2279cb 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1119/out/precommit/personality/provided.sh | | git revision | branch-2 / 12f4e0977c | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/3/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/3/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/3/testReport/ | | Max. process+thread count | 4979 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029529#comment-17029529
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #61 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/61/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/61//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/61//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/61//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/61//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23783) Address tests writing and reading SSL/Security files in a common location.
[
https://issues.apache.org/jira/browse/HBASE-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029512#comment-17029512
]
HBase QA commented on HBASE-23783:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 6 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
3s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
4s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 20m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 2m
53s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
58s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 58s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
25s{color} | {color:green} root: The patch generated 0 new + 0 unchanged - 4
fixed = 0 total (was 4) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 3m
57s{color} | {color:red} patch has 20 errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 2m
9s{color} | {color:red} The patch causes 20 errors with Hadoop v2.8.5. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 4m
23s{color} | {color:red} The patch causes 20 errors with Hadoop v2.9.2. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
41s{color} | {color:red} The patch causes 20 errors with Hadoop v3.1.2. {color}
|
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
55s{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 6m
34s{color} | {color:red} root in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 11m 40s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black}
[GitHub] [hbase] Apache-HBase commented on issue #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
Apache-HBase commented on issue #1116: HBASE-23783: Address tests writing and reading SSL/Security files in … URL: https://github.com/apache/hbase/pull/1116#issuecomment-581721533 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 42s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 43s | master passed | | +1 :green_heart: | compile | 3m 18s | master passed | | +1 :green_heart: | checkstyle | 2m 31s | master passed | | +1 :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 4m 0s | master passed | | +0 :ok: | spotbugs | 4m 4s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 20m 51s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 53s | root in the patch failed. | | -1 :x: | compile | 1m 58s | root in the patch failed. | | -1 :x: | javac | 1m 58s | root in the patch failed. | | +1 :green_heart: | checkstyle | 2m 25s | root: The patch generated 0 new + 0 unchanged - 4 fixed = 0 total (was 4) | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | -1 :x: | shadedjars | 3m 57s | patch has 20 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 2m 9s | The patch causes 20 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 4m 23s | The patch causes 20 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 6m 41s | The patch causes 20 errors with Hadoop v3.1.2. | | +1 :green_heart: | javadoc | 3m 56s | the patch passed | | -1 :x: | findbugs | 0m 55s | hbase-server in the patch failed. | | -1 :x: | findbugs | 6m 34s | root in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 11m 40s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 45s | The patch does not generate ASF License warnings. | | | | 89m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1116 | | JIRA Issue | HBASE-23783 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux fa3fa27d5d76 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1116/out/precommit/personality/provided.sh | | git revision | master / d797ed04e5 | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-compile-root.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-compile-root.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/whitespace-eol.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-shadedjars.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-javac-3.1.2.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1116/2/artifact/out/patch-findbugs-hbase-server.txt | | findbugs |
[jira] [Resolved] (HBASE-23782) We still reference the hard coded meta descriptor in some places when listing table descriptors
[ https://issues.apache.org/jira/browse/HBASE-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-23782. --- Resolution: Fixed Pushed the addendum to master and branch-2. Thanks [~bharathv]. > We still reference the hard coded meta descriptor in some places when listing > table descriptors > --- > > Key: HBASE-23782 > URL: https://issues.apache.org/jira/browse/HBASE-23782 > Project: HBase > Issue Type: Bug > Components: meta >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23782-addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #1115: HBASE-23782 We still reference the hard coded meta descriptor in some…
Apache9 commented on issue #1115: HBASE-23782 We still reference the hard coded meta descriptor in some… URL: https://github.com/apache/hbase/pull/1115#issuecomment-581721293 Pushed an addendum to fix the error prone problem. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] markrmiller opened a new pull request #1120: HBASE-23787: TestSyncTimeRangeTracker fails quite easily and allocate…
markrmiller opened a new pull request #1120: HBASE-23787: TestSyncTimeRangeTracker fails quite easily and allocate… URL: https://github.com/apache/hbase/pull/1120 …s a very expensive array. Here is a first pass, says it can't merge for some reason so I will be back for draft 2. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23787) TestSyncTimeRangeTracker fails quite easily and allocates a very expensive array.
Mark Robert Miller created HBASE-23787: -- Summary: TestSyncTimeRangeTracker fails quite easily and allocates a very expensive array. Key: HBASE-23787 URL: https://issues.apache.org/jira/browse/HBASE-23787 Project: HBase Issue Type: Test Components: test Reporter: Mark Robert Miller I see this test fail a lot in my environments. It also uses such a large array that it seems particularly memory wasteful and difficult to get good contention in the test as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-23782) We still reference the hard coded meta descriptor in some places when listing table descriptors
[ https://issues.apache.org/jira/browse/HBASE-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reopened HBASE-23782: --- Reopen for pushing addendum to fix error prone problem. > We still reference the hard coded meta descriptor in some places when listing > table descriptors > --- > > Key: HBASE-23782 > URL: https://issues.apache.org/jira/browse/HBASE-23782 > Project: HBase > Issue Type: Bug > Components: meta >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23782-addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23782) We still reference the hard coded meta descriptor in some places when listing table descriptors
[ https://issues.apache.org/jira/browse/HBASE-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23782: -- Attachment: HBASE-23782-addendum.patch > We still reference the hard coded meta descriptor in some places when listing > table descriptors > --- > > Key: HBASE-23782 > URL: https://issues.apache.org/jira/browse/HBASE-23782 > Project: HBase > Issue Type: Bug > Components: meta >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-23782-addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029489#comment-17029489
]
Duo Zhang commented on HBASE-23786:
---
I think this is introduced by HBASE-23326. As now we will always create a
HRegion for storing procedures, we will always initialize the ChunkCreator.
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to become active -- which
> doesn't happen when all is easy-going. If struggling under load, it can
> become active and then test asserting NO ChunkCreator instance in Master
> fails because we want ChunkCreator now since ProcedureRegionStore was added:
> i.e. "// always initialize the MemStoreLAB as we use a region to store
> procedure now."
> Here is error I've seen
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] markrmiller commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
markrmiller commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374443660
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestSSLHttpServer.java
##
@@ -51,62 +52,65 @@
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
- HBaseClassTestRule.forClass(TestSSLHttpServer.class);
+HBaseClassTestRule.forClass(TestSSLHttpServer.class);
private static final String BASEDIR = System.getProperty("test.build.dir",
- "target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
+"target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
private static final Logger LOG =
LoggerFactory.getLogger(TestSSLHttpServer.class);
private static Configuration conf;
private static HttpServer server;
private static URL baseUrl;
- private static String keystoresDir;
+ private static File keystoresDir;
private static String sslConfDir;
private static SSLFactory clientSslFactory;
+ private static HBaseCommonTestingUtility htu;
@BeforeClass
public static void setup() throws Exception {
conf = new Configuration();
conf.setInt(HttpServer.HTTP_MAX_THREADS, TestHttpServer.MAX_THREADS);
-File base = new File(BASEDIR);
-FileUtil.fullyDelete(base);
-base.mkdirs();
-keystoresDir = new File(BASEDIR).getAbsolutePath();
+htu = new HBaseCommonTestingUtility(conf);
+
+keystoresDir = new File(htu.getDataTestDir("keystore").toString());
+keystoresDir.mkdirs();
+
sslConfDir = KeyStoreTestUtil.getClasspathDir(TestSSLHttpServer.class);
-KeyStoreTestUtil.setupSSLConfig(keystoresDir, sslConfDir, conf, false);
+KeyStoreTestUtil.setupSSLConfig(keystoresDir.getAbsolutePath(),
sslConfDir, conf, false);
Configuration sslConf = new Configuration(false);
-sslConf.addResource("ssl-server.xml");
-sslConf.addResource("ssl-client.xml");
-
+sslConf.addResource(conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+conf.addResource(conf.get(SSLFactory.SSL_SERVER_CONF_KEY));
+sslConf.set(SSLFactory.SSL_CLIENT_CONF_KEY,
conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+
clientSslFactory = new SSLFactory(SSLFactory.Mode.CLIENT, sslConf);
clientSslFactory.init();
server = new HttpServer.Builder()
-.setName("test")
-.addEndpoint(new URI("https://localhost;))
-.setConf(conf)
-.keyPassword(HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.keypassword",
-null))
-.keyStore(sslConf.get("ssl.server.keystore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.password", null),
-sslConf.get("ssl.server.keystore.type", "jks"))
-.trustStore(sslConf.get("ssl.server.truststore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.truststore.password", null),
-sslConf.get("ssl.server.truststore.type", "jks")).build();
+ .setName("test")
+ .addEndpoint(new URI("https://localhost;))
+ .setConf(conf)
+ .keyPassword(HBaseConfiguration.getPassword(conf,
"ssl.server.keystore.keypassword",
Review comment:
It's more client / server now, tried a rename
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions
Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-581708945 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 58s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 23 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | checkstyle | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 4m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | master passed | | +0 :ok: | spotbugs | 4m 30s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 29s | master passed | | -0 :warning: | patch | 4m 46s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 39s | root in the patch failed. | | -1 :x: | compile | 0m 52s | hbase-server in the patch failed. | | -1 :x: | compile | 0m 26s | hbase-it in the patch failed. | | -1 :x: | javac | 0m 52s | hbase-server in the patch failed. | | -1 :x: | javac | 0m 26s | hbase-it in the patch failed. | | -1 :x: | checkstyle | 1m 15s | hbase-server: The patch generated 14 new + 432 unchanged - 53 fixed = 446 total (was 485) | | -1 :x: | checkstyle | 0m 15s | hbase-it: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -1 :x: | whitespace | 0m 0s | The patch 4 line(s) with tabs. | | -1 :x: | shadedjars | 3m 39s | patch has 15 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 2m 1s | The patch causes 15 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 4m 6s | The patch causes 15 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 6m 15s | The patch causes 15 errors with Hadoop v3.1.2. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | | -1 :x: | findbugs | 0m 50s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 52s | hbase-server in the patch failed. | | -1 :x: | unit | 0m 27s | hbase-it in the patch failed. | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 41m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/921 | | JIRA Issue | HBASE-22749 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e4f51bc0313d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-921/out/precommit/personality/provided.sh | | git revision | master / b591a3d205 | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/patch-compile-hbase-server.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/patch-compile-hbase-it.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/patch-compile-hbase-it.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/diff-checkstyle-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/diff-checkstyle-hbase-it.txt | | whitespace |
[jira] [Updated] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-23786:
--
Fix Version/s: (was: master)
3.0.0
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to become active -- which
> doesn't happen when all is easy-going. If struggling under load, it can
> become active and then test asserting NO ChunkCreator instance in Master
> fails because we want ChunkCreator now since ProcedureRegionStore was added:
> i.e. "// always initialize the MemStoreLAB as we use a region to store
> procedure now."
> Here is error I've seen
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-23786:
--
Description:
Interesting one. Fails only if Master gets chance to become active -- which
doesn't happen when all is easy-going. If struggling under load, it can become
active and then test asserting NO ChunkCreator instance in Master fails because
we want ChunkCreator now since ProcedureRegionStore was added: i.e. "//
always initialize the MemStoreLAB as we use a region to store procedure now."
Here is error I've seen
{code}
[ERROR] Failures:
[ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but
was:
{code}
was:
Interesting one. Fails only if Master gets chance to become active -- which
doesn't happen when all is easy-going. If struggling under load, it can become
active and then test asserting NO ChunkCreator instance in Master fails because
we want ChunkCreator now since ProcedureRegionStore was added.
{code}
[ERROR] Failures:
[ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but
was:
{code}
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0, master
>
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to become active -- which
> doesn't happen when all is easy-going. If struggling under load, it can
> become active and then test asserting NO ChunkCreator instance in Master
> fails because we want ChunkCreator now since ProcedureRegionStore was added:
> i.e. "// always initialize the MemStoreLAB as we use a region to store
> procedure now."
> Here is error I've seen
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-23786:
--
Description:
Interesting one. Fails only if Master gets chance to become active -- which
doesn't happen when all is easy-going. If struggling under load, it can become
active and then test asserting NO ChunkCreator instance in Master fails because
we want ChunkCreator now since ProcedureRegionStore was added.
{code}
[ERROR] Failures:
[ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but
was:
{code}
was:
Interesting one. Fails only if Master gets chance to initialized. It is testing
for ChunkCreator instance in Master. There is supposed to be NONE. But we have
one now since ProcedureRegionStore was added.
{code}
[ERROR] Failures:
[ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but
was:
{code}
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0, master
>
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to become active -- which
> doesn't happen when all is easy-going. If struggling under load, it can
> become active and then test asserting NO ChunkCreator instance in Master
> fails because we want ChunkCreator now since ProcedureRegionStore was added.
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23786.
---
Fix Version/s: master
2.3.0
Assignee: Michael Stack
Resolution: Fixed
Just removed the test that asserts absence of ChunkCreator. Pushed on branch-2
and master.
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0, master
>
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to initialized. It is
> testing for ChunkCreator instance in Master. There is supposed to be NONE.
> But we have one now since ProcedureRegionStore was added.
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
[
https://issues.apache.org/jira/browse/HBASE-23786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-23786:
--
Attachment: 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
> [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
>
>
> Key: HBASE-23786
> URL: https://issues.apache.org/jira/browse/HBASE-23786
> Project: HBase
> Issue Type: Bug
> Components: flakies
>Reporter: Michael Stack
>Priority: Major
> Attachments:
> 0001-HBASE-23786-Flakey-Test-TestMasterNotCarryTable.test.patch
>
>
> Interesting one. Fails only if Master gets chance to initialized. It is
> testing for ChunkCreator instance in Master. There is supposed to be NONE.
> But we have one now since ProcedureRegionStore was added.
> {code}
> [ERROR] Failures:
> [ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null,
> but was:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22834) Remove deprecated methods from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029418#comment-17029418
]
Michael Stack commented on HBASE-22834:
---
Agree. HTU is kinda award in that it is a test utility. W/o, downstreamers
would have hard time doing tests.
> Remove deprecated methods from HBaseTestingUtility
> --
>
> Key: HBASE-22834
> URL: https://issues.apache.org/jira/browse/HBASE-22834
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{HBaseTestingUtility}} has some deprecated methods, which should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23786) [Flakey Test] TestMasterNotCarryTable.testMasterMemStoreLAB
Michael Stack created HBASE-23786:
-
Summary: [Flakey Test]
TestMasterNotCarryTable.testMasterMemStoreLAB
Key: HBASE-23786
URL: https://issues.apache.org/jira/browse/HBASE-23786
Project: HBase
Issue Type: Bug
Components: flakies
Reporter: Michael Stack
Interesting one. Fails only if Master gets chance to initialized. It is testing
for ChunkCreator instance in Master. There is supposed to be NONE. But we have
one now since ProcedureRegionStore was added.
{code}
[ERROR] Failures:
[ERROR] TestMasterNotCarryTable.testMasterMemStoreLAB:94 expected null, but
was:
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23755) [OpenTracing] Declare HTrace is unusable in the user doc
[ https://issues.apache.org/jira/browse/HBASE-23755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23755: - Fix Version/s: (was: 2.3.0) > [OpenTracing] Declare HTrace is unusable in the user doc > > > Key: HBASE-23755 > URL: https://issues.apache.org/jira/browse/HBASE-23755 > Project: HBase > Issue Type: Sub-task >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Fix For: 3.0.0 > > > The trace doesn't work at all in HBase 2.0 and above after HBASE-18601 (the > trace doesn't get picked up at the server side). We should make a note in the > user doc stating it is > # unusable > # deprecated in HBase 2.x because HTrace is in Attic. > # removed from HBase 3.0 and above. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098#issuecomment-581688412 I should say -- I propose one PR per branch containing all the commits, but land it using "rebase and merge" instead of "squash and merge" so that the individual commits are preserved. That make the accounting I refer to simpler. Agreed there's no need for one PR per commit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098#issuecomment-581687963 I see your reasoning. Squishing all the commits makes accounting more difficult, IMHO. The larger issue is that we're still stabilizing the tests vs. master; none of these commits have landed on master yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] apurtell commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
apurtell commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098#issuecomment-581687385 Copying my comment from the JIRA: There were multiple PRs opened for the same set of changes. I merged the one @bharathv indicated. If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth. I think it better one PR/merge for the whole backport per branch, with commit message incorporating all relevant JIRA references. These changes for the most part aren't useful in isolation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (HBASE-23304) Implement RPCs needed for master based registry
[ https://issues.apache.org/jira/browse/HBASE-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029395#comment-17029395 ] Andrew Kyle Purtell edited comment on HBASE-23304 at 2/4/20 12:36 AM: -- There were multiple PRs opened for the same set of changes. I merged the one [~bharathv] indicated at the time. If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth. I think it better one PR/merge for the whole backport per branch, with commit message incorporating all relevant JIRA references. These changes for the most part aren't useful in isolation. was (Author: apurtell): There were multiple PRs opened for the same set of changes. I merged the one [~bharathv] indicated at the time. Going forward, let's have only one PR per branch for backport. If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth. > Implement RPCs needed for master based registry > --- > > Key: HBASE-23304 > URL: https://issues.apache.org/jira/browse/HBASE-23304 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We need to implement RPCs on masters needed by client to fetch information > like clusterID, active master server name, meta locations etc. These RPCs are > used by clients during connection init. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23304) Implement RPCs needed for master based registry
[ https://issues.apache.org/jira/browse/HBASE-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029395#comment-17029395 ] Andrew Kyle Purtell commented on HBASE-23304: - There were multiple PRs opened for the same set of changes. I merged the one [~bharathv] indicated at the time. Going forward, let's have only one PR per branch for backport. If you want to do it per JIRA per branch, fine. One PR per JIRA per branch. However that's a lot of noise for what it's worth. > Implement RPCs needed for master based registry > --- > > Key: HBASE-23304 > URL: https://issues.apache.org/jira/browse/HBASE-23304 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 3.0.0 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > We need to implement RPCs on masters needed by client to fetch information > like clusterID, active master server name, meta locations etc. These RPCs are > used by clients during connection init. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #1113: HBASE-21110 Remove `Illegal reflective access` warning for `java.nio.Bits.unaligned()`
ndimiduk commented on a change in pull request #1113: HBASE-21110 Remove
`Illegal reflective access` warning for `java.nio.Bits.unaligned()`
URL: https://github.com/apache/hbase/pull/1113#discussion_r374414331
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/util/UnsafeAvailChecker.java
##
@@ -21,50 +21,64 @@
import java.lang.reflect.Method;
import java.security.AccessController;
import java.security.PrivilegedAction;
+import sun.misc.Unsafe;
import org.apache.yetus.audience.InterfaceAudience;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+@SuppressWarnings("restriction")
@InterfaceAudience.Private
public class UnsafeAvailChecker {
+ private static final Unsafe UNSAFE;
Review comment:
Indeed that was the old reasoning. The idea was that not all JVM
implementations have an `Unsafe` class, so interfacing with it, if it existed,
had to be done entirely via reflection.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23782) We still reference the hard coded meta descriptor in some places when listing table descriptors
[
https://issues.apache.org/jira/browse/HBASE-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029385#comment-17029385
]
Hudson commented on HBASE-23782:
Results for branch branch-2
[build #2448 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> We still reference the hard coded meta descriptor in some places when listing
> table descriptors
> ---
>
> Key: HBASE-23782
> URL: https://issues.apache.org/jira/browse/HBASE-23782
> Project: HBase
> Issue Type: Bug
> Components: meta
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23780) Edit of test classifications
[
https://issues.apache.org/jira/browse/HBASE-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029386#comment-17029386
]
Hudson commented on HBASE-23780:
Results for branch branch-2
[build #2448 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2448//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Edit of test classifications
>
>
> Key: HBASE-23780
> URL: https://issues.apache.org/jira/browse/HBASE-23780
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Our test classifications have drifted. You can see for yourself running each
> of the small/medium and large test suites. See test complete times. See how
> even some large tests should be small and vice versa.
> The more small tests we can run inside the single JVM, the faster we'll get
> through the build. Tests that are Medium start their own JVM for each test.
> Tests that are Medium but only last a second or two are expensive and should
> be aggregated with other single, short tests to amortize the JVM startup.
> Anyways, let me edit the test categories and try and clean them up some.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22834) Remove deprecated methods from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029384#comment-17029384
]
Nick Dimiduk commented on HBASE-22834:
--
That's a problem. Our tests jars should not be part of our public interface.
> Remove deprecated methods from HBaseTestingUtility
> --
>
> Key: HBASE-22834
> URL: https://issues.apache.org/jira/browse/HBASE-22834
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{HBaseTestingUtility}} has some deprecated methods, which should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on a change in pull request #1115: HBASE-23782 We still reference the hard coded meta descriptor in some…
bharathv commented on a change in pull request #1115: HBASE-23782 We still
reference the hard coded meta descriptor in some…
URL: https://github.com/apache/hbase/pull/1115#discussion_r374407677
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
##
@@ -135,33 +130,30 @@ public FSTableDescriptors(final Configuration conf,
final FileSystem fs,
* see HMaster#finishActiveMasterInitialization
* TODO: This is a workaround. Should remove this ugly
code...
*/
- public FSTableDescriptors(final Configuration conf, final FileSystem fs,
- final Path rootdir, final boolean fsreadonly, final boolean usecache,
- Function metaObserver)
throws IOException {
+ public FSTableDescriptors(final Configuration conf, final FileSystem fs,
final Path rootdir,
+final boolean fsreadonly, final boolean usecache,
+Function metaObserver)
throws IOException {
this.fs = fs;
this.rootdir = rootdir;
this.fsreadonly = fsreadonly;
this.usecache = usecache;
-TableDescriptor td = null;
-try {
- td = getTableDescriptorFromFs(fs, rootdir, TableName.META_TABLE_NAME);
-} catch (TableInfoMissingException e) {
- td = metaObserver == null? createMetaTableDescriptor(conf):
-metaObserver.apply(createMetaTableDescriptorBuilder(conf)).build();
- if (!fsreadonly) {
+if (!fsreadonly) {
+ // see if we already have meta descriptor on fs. Write one if not.
+ try {
+getTableDescriptorFromFs(fs, rootdir, TableName.META_TABLE_NAME);
+ } catch (TableInfoMissingException e) {
+TableDescriptorBuilder builder =
createMetaTableDescriptorBuilder(conf);
+if (metaObserver != null) {
+ metaObserver.apply(builder);
Review comment:
Just FYI, this change broke the error prone build.
https://builds.apache.org/job/HBase%20Nightly/job/master/1618/artifact/output-general/patch-compile-root.txt
[ERROR]
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master@2/component/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java:[147,28]
error: [ReturnValueIgnored] Return value of this method must be used
(see https://errorprone.info/bugpattern/ReturnValueIgnored)
Did you mean to remove this line?
[INFO] 1 error
You can reproduce with "mvn clean compile -DskipTests -PerrorProne"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-23772) Remove deprecated getTimeStampOfLastShippedOp from MetricsSource
[
https://issues.apache.org/jira/browse/HBASE-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated HBASE-23772:
--
Fix Version/s: 3.0.0
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
> Remove deprecated getTimeStampOfLastShippedOp from MetricsSource
>
>
> Key: HBASE-23772
> URL: https://issues.apache.org/jira/browse/HBASE-23772
> Project: HBase
> Issue Type: Task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> {{MetricsSource}} defines the deprecated method
> {{getTimeStampOfLastShippedOp}}, which should be removed for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] bharathv commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
bharathv commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119#issuecomment-581669109 +1 subject to a clean QA build. Thanks for the revert. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet merged pull request #1107: HBASE-23772 Removed deprecated getTimeStampOfLastShippedOp from MetricsSource
HorizonNet merged pull request #1107: HBASE-23772 Removed deprecated getTimeStampOfLastShippedOp from MetricsSource URL: https://github.com/apache/hbase/pull/1107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119#issuecomment-581660995 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 12s | https://github.com/apache/hbase/pull/1119 does not apply to branch-2. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bharathv commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
bharathv commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098#issuecomment-581657310 Ya, so this is a squashed commit up until HBASE-23304. I'm ok with reverting it. I can submit a single PR with all the commits later, once the feature branch merges with master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23304) Implement RPCs needed for master based registry
[
https://issues.apache.org/jira/browse/HBASE-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029328#comment-17029328
]
Sean Busbey commented on HBASE-23304:
-
Comment on PR #1098 about that branch-2 PR
{quote}
bharath apurtell What exactly was this commit? The message looks like it
contains everything from the feature branch, but the changed files is just
HBASE-23304.
Anyway, we haven't merged the feature branch to master yet; we're not quite
ready to back port to branch-2. I think this should be reverted.
{quote}
Revert is PR #1119
> Implement RPCs needed for master based registry
> ---
>
> Key: HBASE-23304
> URL: https://issues.apache.org/jira/browse/HBASE-23304
> Project: HBase
> Issue Type: Sub-task
> Components: master
>Affects Versions: 3.0.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> We need to implement RPCs on masters needed by client to fetch information
> like clusterID, active master server name, meta locations etc. These RPCs are
> used by clients during connection init.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
Apache-HBase commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119#issuecomment-581641300 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 1s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 18 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 30s | branch-2 passed | | +1 :green_heart: | compile | 2m 26s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 17s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | branch-2 passed | | +0 :ok: | spotbugs | 0m 38s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 8m 13s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 49s | root in the patch failed. | | -1 :x: | compile | 0m 55s | hbase-server in the patch failed. | | -1 :x: | cc | 0m 55s | hbase-server in the patch failed. | | -1 :x: | javac | 0m 55s | hbase-server in the patch failed. | | -1 :x: | checkstyle | 0m 36s | hbase-client: The patch generated 1 new + 194 unchanged - 0 fixed = 195 total (was 194) | | -1 :x: | checkstyle | 1m 8s | hbase-server: The patch generated 13 new + 119 unchanged - 0 fixed = 132 total (was 119) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | shadedjars | 3m 34s | patch has 36 errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 2m 5s | The patch causes 36 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 4m 35s | The patch causes 36 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 7m 31s | The patch causes 36 errors with Hadoop v3.1.2. | | -1 :x: | hbaseprotoc | 0m 55s | hbase-server in the patch failed. | | +1 :green_heart: | javadoc | 1m 35s | the patch passed | | -1 :x: | findbugs | 1m 1s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 50s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 4m 3s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 53s | hbase-zookeeper in the patch passed. | | -1 :x: | unit | 1m 5s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. | | | | 64m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1119 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 2d3305e3a61c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1119/out/precommit/personality/provided.sh | | git revision | branch-2 / 12f4e0977c | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/patch-compile-hbase-server.txt | | cc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/patch-compile-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1119/1/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle |
[jira] [Resolved] (HBASE-23763) Add 'new on release line report' for git/jira audit tool
[
https://issues.apache.org/jira/browse/HBASE-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23763.
--
Resolution: Fixed
> Add 'new on release line report' for git/jira audit tool
>
>
> Key: HBASE-23763
> URL: https://issues.apache.org/jira/browse/HBASE-23763
> Project: HBase
> Issue Type: Task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0
>
>
> Codify building the summary of what's new on a release line branch (i.e.,
> {{branch-2}}), but not yet released on earlier release branches of that line.
> Builds a cvs report that looks like
> [https://home.apache.org/~ndimiduk/new_for_branch-2.csv|https://home.apache.org/~ndimiduk/new_for_branch-2.csv]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23785) Update docs re: Hadoop compatibility of 2.3.x release
[ https://issues.apache.org/jira/browse/HBASE-23785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23785: - Fix Version/s: 2.3.0 3.0.0 Setting fixVersion to 2.3.0 so the issue is easily tracking along side the release. > Update docs re: Hadoop compatibility of 2.3.x release > - > > Key: HBASE-23785 > URL: https://issues.apache.org/jira/browse/HBASE-23785 > Project: HBase > Issue Type: Task > Components: documentation >Reporter: Nick Dimiduk >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Update the [book|http://hbase.apache.org/book.html#hadoop] regarding our > Hadoop compatibility for the 2.3 release lines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23785) Update docs re: Hadoop compatibility of 2.3.x release
Nick Dimiduk created HBASE-23785: Summary: Update docs re: Hadoop compatibility of 2.3.x release Key: HBASE-23785 URL: https://issues.apache.org/jira/browse/HBASE-23785 Project: HBase Issue Type: Task Components: documentation Reporter: Nick Dimiduk Update the [book|http://hbase.apache.org/book.html#hadoop] regarding our Hadoop compatibility for the 2.3 release lines. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
ndimiduk commented on issue #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119#issuecomment-581613155 FYI @bharathv @apurtell This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk opened a new pull request #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (…
ndimiduk opened a new pull request #1119: Revert "HBASE-23304: RPCs needed for client meta information lookup (… URL: https://github.com/apache/hbase/pull/1119 …apache#904) (#1098)" This reverts commit 71f035450df0c8f56622b5151003bc134ca0091b. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904)
ndimiduk commented on issue #1098: HBASE-23304: RPCs needed for client meta information lookup (apache#904) URL: https://github.com/apache/hbase/pull/1098#issuecomment-581611214 @bharathv @apurtell What exactly was this commit? The message looks like it contains everything from the feature branch, but the changed files is just HBASE-23304. Anyway, we haven't merged the feature branch to master yet; we're not quite ready to back port to branch-2. I think this should be reverted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23780) Edit of test classifications
[ https://issues.apache.org/jira/browse/HBASE-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23780: - Fix Version/s: (was: master) 3.0.0 > Edit of test classifications > > > Key: HBASE-23780 > URL: https://issues.apache.org/jira/browse/HBASE-23780 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > Our test classifications have drifted. You can see for yourself running each > of the small/medium and large test suites. See test complete times. See how > even some large tests should be small and vice versa. > The more small tests we can run inside the single JVM, the faster we'll get > through the build. Tests that are Medium start their own JVM for each test. > Tests that are Medium but only last a second or two are expensive and should > be aggregated with other single, short tests to amortize the JVM startup. > Anyways, let me edit the test categories and try and clean them up some. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23350) Make compaction files cacheonWrite configurable based on threshold
[ https://issues.apache.org/jira/browse/HBASE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029246#comment-17029246 ] Nick Dimiduk commented on HBASE-23350: -- Looks like this hit branch-2. Adding the fixVersion. > Make compaction files cacheonWrite configurable based on threshold > -- > > Key: HBASE-23350 > URL: https://issues.apache.org/jira/browse/HBASE-23350 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: ramkrishna.s.vasudevan >Assignee: Abhinaba Sarkar >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > As per comment from [~javaman_chen] in the parent JIRA > https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16937361=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937361 > This is to introduce a config to identify if the resulting compacted file's > blocks should be added to the cache - while writing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23350) Make compaction files cacheonWrite configurable based on threshold
[ https://issues.apache.org/jira/browse/HBASE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23350: - Fix Version/s: 2.3.0 > Make compaction files cacheonWrite configurable based on threshold > -- > > Key: HBASE-23350 > URL: https://issues.apache.org/jira/browse/HBASE-23350 > Project: HBase > Issue Type: Sub-task > Components: Compaction >Reporter: ramkrishna.s.vasudevan >Assignee: Abhinaba Sarkar >Priority: Major > Fix For: 3.0.0, 2.3.0 > > > As per comment from [~javaman_chen] in the parent JIRA > https://issues.apache.org/jira/browse/HBASE-23066?focusedCommentId=16937361=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16937361 > This is to introduce a config to identify if the resulting compacted file's > blocks should be added to the cache - while writing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-23783) Address tests writing and reading SSL/Security files in a common location.
[
https://issues.apache.org/jira/browse/HBASE-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029230#comment-17029230
]
Mark Robert Miller edited comment on HBASE-23783 at 2/3/20 8:04 PM:
I would also like to add ${surefire.tempDir}
To safely run surefire from multiple maven instances, you have to be able to
specify a unique tmp directory. Otherwise, removal of the directory on JVM exit
can interfere with tmp file creation.
was (Author: markrmiller):
I also like to add ${surefire.tempDir}
To safely run surefire from multiple maven instances, you have to be able to
specify a unique tmp directory.
> Address tests writing and reading SSL/Security files in a common location.
> --
>
> Key: HBASE-23783
> URL: https://issues.apache.org/jira/browse/HBASE-23783
> Project: HBase
> Issue Type: Test
>Reporter: Mark Robert Miller
>Priority: Minor
>
> This is causing me issues with parallel test runs because multiple tests can
> write and read the same files in the test-classes directory. Some tests write
> files in test-classes instead of their test data directory so that they can
> put the files on the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23783) Address tests writing and reading SSL/Security files in a common location.
[
https://issues.apache.org/jira/browse/HBASE-23783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029230#comment-17029230
]
Mark Robert Miller commented on HBASE-23783:
I also like to add ${surefire.tempDir}
To safely run surefire from multiple maven instances, you have to be able to
specify a unique tmp directory.
> Address tests writing and reading SSL/Security files in a common location.
> --
>
> Key: HBASE-23783
> URL: https://issues.apache.org/jira/browse/HBASE-23783
> Project: HBase
> Issue Type: Test
>Reporter: Mark Robert Miller
>Priority: Minor
>
> This is causing me issues with parallel test runs because multiple tests can
> write and read the same files in the test-classes directory. Some tests write
> files in test-classes instead of their test data directory so that they can
> put the files on the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23740) Invalid StoreFile WARN log message printed for recovered.hfiles directory
[
https://issues.apache.org/jira/browse/HBASE-23740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029199#comment-17029199
]
Pankaj Kumar commented on HBASE-23740:
--
{quote}So ya we should have a mechanism to exclude this path (some regex way)
like we exclude the recovered.edits under region dir.
{quote}
Make sense, will raise PR accordingly.
{quote}Good catch. So u r testing the cluster restart and all RS WAL split
cases here. Good
{quote}
Yeah, we backported this to branch-1 and verifying all the impacted features.
> Invalid StoreFile WARN log message printed for recovered.hfiles directory
> -
>
> Key: HBASE-23740
> URL: https://issues.apache.org/jira/browse/HBASE-23740
> Project: HBase
> Issue Type: Sub-task
> Components: MTTR
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Pankaj Kumar
>Priority: Minor
>
> After HBASE-23286, recovered.hfiles directories will be created under table
> CF dir. Region server validates the store file during store initializaiton.
> Currently it expect only store file inside the table CF directory, so for
> recovered.hfiles below WARN logs message will be printed.
> {code:java}
> 2020-01-27 11:13:09,823 WARN [StoreOpener-1588230740-1]
> regionserver.HRegionFileSystem: Invalid StoreFile:
> hdfs://127.0.0.1:8020/hbase/data/hbase/meta/1588230740/info/recovered.hfiles
> 2020-01-27 11:14:31,036 WARN [StoreOpener-1588230740-1]
> regionserver.HRegionFileSystem: Invalid StoreFile:
> hdfs://127.0.0.1:8020/hbase/data/hbase/meta/1588230740/info/recovered.hfiles
> 2020-01-27 11:14:31,110 WARN [StoreOpener-1588230740-1]
> regionserver.HRegionFileSystem: Invalid StoreFile:
> hdfs://127.0.0.1:8020/hbase/data/hbase/meta/1588230740/table/recovered.hfiles
> 2020-01-27 11:14:36,031 WARN [StoreOpener-16beeb2fd517283980b11b912490cbba-1]
> regionserver.HRegionFileSystem: Invalid StoreFile:
> hdfs://127.0.0.1:8020/hbase/data/default/t4/16beeb2fd517283980b11b912490cbba/cf1/recovered.hfiles
> 2020-01-27 11:14:36,057 WARN [StoreOpener-e430213f4660304a26b9b389f78b5681-1]
> regionserver.HRegionFileSystem: Invalid StoreFile:
> hdfs://127.0.0.1:8020/hbase/data/hbase/acl/e430213f4660304a26b9b389f78b5681/l/recovered.hfiles
> {code}
> No need to validate recovered.hfiles directory.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
saintstack commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374273924
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestSSLHttpServer.java
##
@@ -51,62 +52,65 @@
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
- HBaseClassTestRule.forClass(TestSSLHttpServer.class);
+HBaseClassTestRule.forClass(TestSSLHttpServer.class);
private static final String BASEDIR = System.getProperty("test.build.dir",
- "target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
+"target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
private static final Logger LOG =
LoggerFactory.getLogger(TestSSLHttpServer.class);
private static Configuration conf;
private static HttpServer server;
private static URL baseUrl;
- private static String keystoresDir;
+ private static File keystoresDir;
private static String sslConfDir;
private static SSLFactory clientSslFactory;
+ private static HBaseCommonTestingUtility htu;
Review comment:
Statics. normally capitalized as in HTU rather than htu.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
saintstack commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374274553
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestSSLHttpServer.java
##
@@ -51,62 +52,65 @@
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
- HBaseClassTestRule.forClass(TestSSLHttpServer.class);
+HBaseClassTestRule.forClass(TestSSLHttpServer.class);
private static final String BASEDIR = System.getProperty("test.build.dir",
- "target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
+"target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
private static final Logger LOG =
LoggerFactory.getLogger(TestSSLHttpServer.class);
private static Configuration conf;
private static HttpServer server;
private static URL baseUrl;
- private static String keystoresDir;
+ private static File keystoresDir;
private static String sslConfDir;
private static SSLFactory clientSslFactory;
+ private static HBaseCommonTestingUtility htu;
@BeforeClass
public static void setup() throws Exception {
conf = new Configuration();
conf.setInt(HttpServer.HTTP_MAX_THREADS, TestHttpServer.MAX_THREADS);
-File base = new File(BASEDIR);
-FileUtil.fullyDelete(base);
-base.mkdirs();
-keystoresDir = new File(BASEDIR).getAbsolutePath();
+htu = new HBaseCommonTestingUtility(conf);
+
+keystoresDir = new File(htu.getDataTestDir("keystore").toString());
+keystoresDir.mkdirs();
+
sslConfDir = KeyStoreTestUtil.getClasspathDir(TestSSLHttpServer.class);
-KeyStoreTestUtil.setupSSLConfig(keystoresDir, sslConfDir, conf, false);
+KeyStoreTestUtil.setupSSLConfig(keystoresDir.getAbsolutePath(),
sslConfDir, conf, false);
Configuration sslConf = new Configuration(false);
-sslConf.addResource("ssl-server.xml");
-sslConf.addResource("ssl-client.xml");
-
+sslConf.addResource(conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+conf.addResource(conf.get(SSLFactory.SSL_SERVER_CONF_KEY));
+sslConf.set(SSLFactory.SSL_CLIENT_CONF_KEY,
conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+
clientSslFactory = new SSLFactory(SSLFactory.Mode.CLIENT, sslConf);
clientSslFactory.init();
server = new HttpServer.Builder()
-.setName("test")
-.addEndpoint(new URI("https://localhost;))
-.setConf(conf)
-.keyPassword(HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.keypassword",
-null))
-.keyStore(sslConf.get("ssl.server.keystore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.password", null),
-sslConf.get("ssl.server.keystore.type", "jks"))
-.trustStore(sslConf.get("ssl.server.truststore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.truststore.password", null),
-sslConf.get("ssl.server.truststore.type", "jks")).build();
+ .setName("test")
+ .addEndpoint(new URI("https://localhost;))
+ .setConf(conf)
+ .keyPassword(HBaseConfiguration.getPassword(conf,
"ssl.server.keystore.keypassword",
Review comment:
Above its sslconf... you want conf here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
saintstack commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374274936
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/TestSSLHttpServer.java
##
@@ -51,62 +52,65 @@
@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
- HBaseClassTestRule.forClass(TestSSLHttpServer.class);
+HBaseClassTestRule.forClass(TestSSLHttpServer.class);
private static final String BASEDIR = System.getProperty("test.build.dir",
- "target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
+"target/test-dir") + "/" + TestSSLHttpServer.class.getSimpleName();
private static final Logger LOG =
LoggerFactory.getLogger(TestSSLHttpServer.class);
private static Configuration conf;
private static HttpServer server;
private static URL baseUrl;
- private static String keystoresDir;
+ private static File keystoresDir;
private static String sslConfDir;
private static SSLFactory clientSslFactory;
+ private static HBaseCommonTestingUtility htu;
@BeforeClass
public static void setup() throws Exception {
conf = new Configuration();
conf.setInt(HttpServer.HTTP_MAX_THREADS, TestHttpServer.MAX_THREADS);
-File base = new File(BASEDIR);
-FileUtil.fullyDelete(base);
-base.mkdirs();
-keystoresDir = new File(BASEDIR).getAbsolutePath();
+htu = new HBaseCommonTestingUtility(conf);
+
+keystoresDir = new File(htu.getDataTestDir("keystore").toString());
+keystoresDir.mkdirs();
+
sslConfDir = KeyStoreTestUtil.getClasspathDir(TestSSLHttpServer.class);
-KeyStoreTestUtil.setupSSLConfig(keystoresDir, sslConfDir, conf, false);
+KeyStoreTestUtil.setupSSLConfig(keystoresDir.getAbsolutePath(),
sslConfDir, conf, false);
Configuration sslConf = new Configuration(false);
-sslConf.addResource("ssl-server.xml");
-sslConf.addResource("ssl-client.xml");
-
+sslConf.addResource(conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+conf.addResource(conf.get(SSLFactory.SSL_SERVER_CONF_KEY));
+sslConf.set(SSLFactory.SSL_CLIENT_CONF_KEY,
conf.get(SSLFactory.SSL_CLIENT_CONF_KEY));
+
clientSslFactory = new SSLFactory(SSLFactory.Mode.CLIENT, sslConf);
clientSslFactory.init();
server = new HttpServer.Builder()
-.setName("test")
-.addEndpoint(new URI("https://localhost;))
-.setConf(conf)
-.keyPassword(HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.keypassword",
-null))
-.keyStore(sslConf.get("ssl.server.keystore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.keystore.password", null),
-sslConf.get("ssl.server.keystore.type", "jks"))
-.trustStore(sslConf.get("ssl.server.truststore.location"),
-HBaseConfiguration.getPassword(sslConf,
"ssl.server.truststore.password", null),
-sslConf.get("ssl.server.truststore.type", "jks")).build();
+ .setName("test")
+ .addEndpoint(new URI("https://localhost;))
+ .setConf(conf)
+ .keyPassword(HBaseConfiguration.getPassword(conf,
"ssl.server.keystore.keypassword",
+null))
+ .keyStore(conf.get("ssl.server.keystore.location"),
Review comment:
Ditto. We use conf instead of sslconf here... That intentional?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
saintstack commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374275731
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/log/TestLogLevel.java
##
@@ -84,31 +83,29 @@
private final static String KEYTAB = "loglevel.keytab";
private static MiniKdc kdc;
- private static HBaseCommonTestingUtility htu = new
HBaseCommonTestingUtility();
private static final String LOCALHOST = "localhost";
private static final String clientPrincipal = "client/" + LOCALHOST;
private static String HTTP_PRINCIPAL = "HTTP/" + LOCALHOST;
-
- private static final File KEYTAB_FILE = new File(
- htu.getDataTestDir("keytab").toUri().getPath());
+ private static HBaseCommonTestingUtility htu;
Review comment:
Capitalize
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1116: HBASE-23783: Address tests writing and reading SSL/Security files in …
saintstack commented on a change in pull request #1116: HBASE-23783: Address
tests writing and reading SSL/Security files in …
URL: https://github.com/apache/hbase/pull/1116#discussion_r374276429
##
File path:
hbase-http/src/test/java/org/apache/hadoop/hbase/http/log/TestLogLevel.java
##
@@ -84,31 +83,29 @@
private final static String KEYTAB = "loglevel.keytab";
private static MiniKdc kdc;
- private static HBaseCommonTestingUtility htu = new
HBaseCommonTestingUtility();
private static final String LOCALHOST = "localhost";
private static final String clientPrincipal = "client/" + LOCALHOST;
private static String HTTP_PRINCIPAL = "HTTP/" + LOCALHOST;
-
- private static final File KEYTAB_FILE = new File(
- htu.getDataTestDir("keytab").toUri().getPath());
+ private static HBaseCommonTestingUtility htu;
+ private static File keyTabFile;
@BeforeClass
public static void setUp() throws Exception {
-BASEDIR = new File(htu.getDataTestDir().toUri().getPath());
-
-FileUtil.fullyDelete(BASEDIR);
-if (!BASEDIR.mkdirs()) {
- throw new Exception("unable to create the base directory for testing");
-}
serverConf = new Configuration();
+htu = new HBaseCommonTestingUtility(serverConf);
Review comment:
And maybe you want to use the HTU's conf. Do HTU.getConfiguration instead of
creating a 'new Configuration()' in line above ... and FYI, for configs that
will include hbase config, you want HBaseConfiguration.create instead of
new Configuration.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23633) Find a way to handle the corrupt recovered hfiles
[ https://issues.apache.org/jira/browse/HBASE-23633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029184#comment-17029184 ] Pankaj Kumar commented on HBASE-23633: -- Will raise PR with UT. > Find a way to handle the corrupt recovered hfiles > - > > Key: HBASE-23633 > URL: https://issues.apache.org/jira/browse/HBASE-23633 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Priority: Major > > Copy the comment from PR review. > > If the file is a corrupt HFile, an exception will be thrown here, which will > cause the region to fail to open. > Maybe we can add a new parameter to control whether to skip the exception, > similar to recover edits which has a parameter > "hbase.hregion.edits.replay.skip.errors"; > > Regions that can't be opened because of detached References or corrupt hfiles > are a fact-of-life. We need work on this issue. This will be a new variant on > the problem -- i.e. bad recovered hfiles. > On adding a config to ignore bad files and just open, thats a bit dangerous > as per @infraio as it could mean silent data loss. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029176#comment-17029176
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #60 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/60/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/60//console].
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/60//console].
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- Something went wrong running this stage, please [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/60//console].
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/60//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] VladRodionov opened a new pull request #921: HBASE-22749: Distributed MOB compactions
VladRodionov opened a new pull request #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921 Finalizing HBASE-22749. Rebased patch against current HBase master, created new pull PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23780) Edit of test classifications
[
https://issues.apache.org/jira/browse/HBASE-23780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack resolved HBASE-23780.
---
Fix Version/s: master
2.3.0
Hadoop Flags: Reviewed
Assignee: Michael Stack
Resolution: Fixed
Pushed this on branch-2 and master. Reclassifies tests according to how long
they take in recent runs.
Idea is to move as many tests as possible into small category where they all
run in the one JVM. Allows us to move through a bunch in a short time.
Next up is getting tests into the medium category. These are tests that take <
50 seconds. Parallelization should help here as they don't stick around too
long. TODO is fix tests that only run a single test and the test takes little
time... Aggregate up single tests into one test suite. Medium tests start a
JVM. This costs. Amortize.
Large tests we'll just have to tackle one at a time.
I've been using a coworker's script to up the parallelization:
{code}$ x=1.0 mvn -T $xC -Dsurefire.firstPartForkCount=$xC
-Dsurefire.secondPartForkCount=$xC -PrunSmallTests{code} ... etc.
The -T in particular helps by making maven use more than one thread.
> Edit of test classifications
>
>
> Key: HBASE-23780
> URL: https://issues.apache.org/jira/browse/HBASE-23780
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 2.3.0, master
>
>
> Our test classifications have drifted. You can see for yourself running each
> of the small/medium and large test suites. See test complete times. See how
> even some large tests should be small and vice versa.
> The more small tests we can run inside the single JVM, the faster we'll get
> through the build. Tests that are Medium start their own JVM for each test.
> Tests that are Medium but only last a second or two are expensive and should
> be aggregated with other single, short tests to amortize the JVM startup.
> Anyways, let me edit the test categories and try and clean them up some.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1118: HBASE-23276 Add admin methods to get tables within a group
Apache-HBase commented on issue #1118: HBASE-23276 Add admin methods to get tables within a group URL: https://github.com/apache/hbase/pull/1118#issuecomment-581549565 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | +0 :ok: | mvndep | 0m 37s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 20s | HBASE-22514 passed | | +1 :green_heart: | compile | 2m 38s | HBASE-22514 passed | | +1 :green_heart: | checkstyle | 2m 28s | HBASE-22514 passed | | +1 :green_heart: | shadedjars | 4m 45s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 47s | HBASE-22514 passed | | +0 :ok: | spotbugs | 1m 32s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 10m 13s | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 4m 45s | root in the patch failed. | | +1 :green_heart: | compile | 2m 36s | the patch passed | | +1 :green_heart: | cc | 2m 36s | the patch passed | | +1 :green_heart: | javac | 2m 36s | the patch passed | | -1 :x: | checkstyle | 0m 30s | hbase-client: The patch generated 5 new + 50 unchanged - 0 fixed = 55 total (was 50) | | -1 :x: | checkstyle | 1m 7s | hbase-server: The patch generated 3 new + 68 unchanged - 0 fixed = 71 total (was 68) | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 2m 53s | The patch causes 16 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 5m 59s | The patch causes 16 errors with Hadoop v2.9.2. | | -1 :x: | hadoopcheck | 9m 13s | The patch causes 16 errors with Hadoop v3.1.2. | | +1 :green_heart: | hbaseprotoc | 2m 28s | the patch passed | | +1 :green_heart: | javadoc | 1m 47s | the patch passed | | +1 :green_heart: | findbugs | 10m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 47s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 4s | hbase-client in the patch passed. | | -1 :x: | unit | 146m 37s | hbase-server in the patch failed. | | -1 :x: | unit | 3m 10s | hbase-thrift in the patch failed. | | +1 :green_heart: | asflicense | 1m 42s | The patch does not generate ASF License warnings. | | | | 224m 33s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.security.access.TestAccessController3 | | | hadoop.hbase.thrift2.TestThriftHBaseServiceHandlerWithReadOnly | | | hadoop.hbase.thrift.TestThriftServer | | | hadoop.hbase.thrift2.TestThriftHBaseServiceHandlerWithLabels | | | hadoop.hbase.thrift2.TestThriftHBaseServiceHandler | | | hadoop.hbase.thrift.TestThriftSpnegoHttpServer | | | hadoop.hbase.thrift2.TestThriftConnection | | | hadoop.hbase.thrift2.TestThrift2HttpServer | | | hadoop.hbase.thrift.TestThriftSpnegoHttpFallbackServer | | | hadoop.hbase.thrift2.TestThrift2ServerCmdLine | | | hadoop.hbase.thrift.TestThriftServerCmdLine | | | hadoop.hbase.thrift.TestThriftHttpServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1118/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1118 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 53cc4b985318 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1118/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 31785962af | | Default Java | 1.8.0_181 | | mvninstall |
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23752: - Resolution: Fixed Status: Resolved (was: Patch Available) > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: HBASE-18095 > > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found the > following issues. > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > TestSnapshotScannerHDFSAclController: Cluster restart did not reset the > cached connection state > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23752) Fix a couple more test failures from nightly run
[ https://issues.apache.org/jira/browse/HBASE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23752: - Fix Version/s: HBASE-18095 > Fix a couple more test failures from nightly run > > > Key: HBASE-23752 > URL: https://issues.apache.org/jira/browse/HBASE-23752 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0, 2.3.0, 1.6.0, HBASE-18095 >Reporter: Bharath Vissapragada >Assignee: Bharath Vissapragada >Priority: Major > Fix For: HBASE-18095 > > > After merging the patches HBASE-23731 and HBASE-23647, nightly run found the > following issues. > TestFromClientSideWithCoprocessor - This was a bug in the test itself, PR > runs missed it somehow. > TestCustomSaslAuthenticationProvider - This was written pre master registry. > Has to be updated. It was never run as a part of the test suite. > TestSnapshotScannerHDFSAclController: Cluster restart did not reset the > cached connection state > > [~ndimiduk] FYI because this is a blocker for 2.3.0 too, I tagged it > accordingly. Will submit a PR shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs
ndimiduk merged pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs URL: https://github.com/apache/hbase/pull/1102 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs
ndimiduk commented on a change in pull request #1102: HBASE-23752: Fix
remaining test failures from nightly runs
URL: https://github.com/apache/hbase/pull/1102#discussion_r374257313
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/security/provider/TestCustomSaslAuthenticationProvider.java
##
@@ -443,9 +464,23 @@ public static void setupCluster() throws Exception {
InMemoryServerProvider.class.getName());
CONF.set(SaslClientAuthenticationProviders.SELECTOR_KEY,
InMemoryProviderSelector.class.getName());
+createBaseCluster(UTIL, KEYTAB_FILE, kdc);
+ }
-CLUSTER = createCluster(UTIL, KEYTAB_FILE, kdc);
+ @Before
+ public void setUpBeforeTest() throws Exception {
+CONF.unset(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY);
+CONF.set(RpcClientFactory.CUSTOM_RPC_CLIENT_IMPL_CONF_KEY, rpcClientImpl);
+CONF.set(RpcServerFactory.CUSTOM_RPC_SERVER_IMPL_CONF_KEY, rpcServerImpl);
+if (rpcClientImpl.equals(BlockingRpcClient.class.getName())) {
+ // Set the connection registry to ZKConnectionRegistry since hedging is
not supported on
+ // blocking rpc clients.
+ CONF.set(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY,
+ HConstants.ZK_CONNECTION_REGISTRY_CLASS);
+}
+CLUSTER = new LocalHBaseCluster(CONF, 1);
Review comment:
Okay, makes sense. Thanks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] bharathv commented on a change in pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs
bharathv commented on a change in pull request #1102: HBASE-23752: Fix
remaining test failures from nightly runs
URL: https://github.com/apache/hbase/pull/1102#discussion_r374255049
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/security/provider/TestCustomSaslAuthenticationProvider.java
##
@@ -443,9 +464,23 @@ public static void setupCluster() throws Exception {
InMemoryServerProvider.class.getName());
CONF.set(SaslClientAuthenticationProviders.SELECTOR_KEY,
InMemoryProviderSelector.class.getName());
+createBaseCluster(UTIL, KEYTAB_FILE, kdc);
+ }
-CLUSTER = createCluster(UTIL, KEYTAB_FILE, kdc);
+ @Before
+ public void setUpBeforeTest() throws Exception {
+CONF.unset(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY);
+CONF.set(RpcClientFactory.CUSTOM_RPC_CLIENT_IMPL_CONF_KEY, rpcClientImpl);
+CONF.set(RpcServerFactory.CUSTOM_RPC_SERVER_IMPL_CONF_KEY, rpcServerImpl);
+if (rpcClientImpl.equals(BlockingRpcClient.class.getName())) {
+ // Set the connection registry to ZKConnectionRegistry since hedging is
not supported on
+ // blocking rpc clients.
+ CONF.set(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY,
+ HConstants.ZK_CONNECTION_REGISTRY_CLASS);
+}
+CLUSTER = new LocalHBaseCluster(CONF, 1);
Review comment:
Ya, unfortunately its the same problem, JUnit calls the ctor for every
parameterized test invocation. So we'd need to have some logic like
if (same instance of params) don't restart
We did that in TestFromClientSide but the code becomes ugly. Lucikly, there
are fewer tests, so this works out ok here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ndimiduk commented on a change in pull request #1102: HBASE-23752: Fix remaining test failures from nightly runs
ndimiduk commented on a change in pull request #1102: HBASE-23752: Fix
remaining test failures from nightly runs
URL: https://github.com/apache/hbase/pull/1102#discussion_r374249918
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/security/provider/TestCustomSaslAuthenticationProvider.java
##
@@ -443,9 +464,23 @@ public static void setupCluster() throws Exception {
InMemoryServerProvider.class.getName());
CONF.set(SaslClientAuthenticationProviders.SELECTOR_KEY,
InMemoryProviderSelector.class.getName());
+createBaseCluster(UTIL, KEYTAB_FILE, kdc);
+ }
-CLUSTER = createCluster(UTIL, KEYTAB_FILE, kdc);
+ @Before
+ public void setUpBeforeTest() throws Exception {
+CONF.unset(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY);
+CONF.set(RpcClientFactory.CUSTOM_RPC_CLIENT_IMPL_CONF_KEY, rpcClientImpl);
+CONF.set(RpcServerFactory.CUSTOM_RPC_SERVER_IMPL_CONF_KEY, rpcServerImpl);
+if (rpcClientImpl.equals(BlockingRpcClient.class.getName())) {
+ // Set the connection registry to ZKConnectionRegistry since hedging is
not supported on
+ // blocking rpc clients.
+ CONF.set(HConstants.CLIENT_CONNECTION_REGISTRY_IMPL_CONF_KEY,
+ HConstants.ZK_CONNECTION_REGISTRY_CLASS);
+}
+CLUSTER = new LocalHBaseCluster(CONF, 1);
Review comment:
Sucks that we have to spin the new cluster for every test method. It is not
possible to do this cluster configuration and initialization in the constructor
instead?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22834) Remove deprecated methods from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029131#comment-17029131
]
Michael Stack commented on HBASE-22834:
---
bq. Yikes, why is HBaseTestingUtility considered a Public interface?!
I think its because it is used by downstreamers.
> Remove deprecated methods from HBaseTestingUtility
> --
>
> Key: HBASE-22834
> URL: https://issues.apache.org/jira/browse/HBASE-22834
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{HBaseTestingUtility}} has some deprecated methods, which should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #1108: HBASE-23779 Up the default fork count; make count relative to CPU count
ndimiduk commented on issue #1108: HBASE-23779 Up the default fork count; make count relative to CPU count URL: https://github.com/apache/hbase/pull/1108#issuecomment-581530433 > Interesting. I got this failure: > > [ERROR] Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? > > ... which I've seen running with higher values of -T/forkCount on virtualbox. I've seen similar running on OSX as well. No root cause as of yet. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22834) Remove deprecated methods from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-22834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029118#comment-17029118
]
Nick Dimiduk commented on HBASE-22834:
--
Yikes, why is HBaseTestingUtility considered a Public interface?! This class is
under the {{src/tests}} directory; it has no business being consumed by any
code outside of this project.
> Remove deprecated methods from HBaseTestingUtility
> --
>
> Key: HBASE-22834
> URL: https://issues.apache.org/jira/browse/HBASE-22834
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{HBaseTestingUtility}} has some deprecated methods, which should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23775) Remove deprecated createLocalHTU(Configuration) from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-23775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029116#comment-17029116
]
Nick Dimiduk commented on HBASE-23775:
--
Actually let me take this question to the parent ticket.
> Remove deprecated createLocalHTU(Configuration) from HBaseTestingUtility
>
>
> Key: HBASE-23775
> URL: https://issues.apache.org/jira/browse/HBASE-23775
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{createLocalHTU(Configuration)}} in {{HBaseTestingUtility}} was deprecated
> back in 2.0.0 and should be removed for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23775) Remove deprecated createLocalHTU(Configuration) from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-23775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029114#comment-17029114
]
Nick Dimiduk commented on HBASE-23775:
--
Yikes, why is HBaseTestingUtility considered a Public interface?!
> Remove deprecated createLocalHTU(Configuration) from HBaseTestingUtility
>
>
> Key: HBASE-23775
> URL: https://issues.apache.org/jira/browse/HBASE-23775
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
>
> {{createLocalHTU(Configuration)}} in {{HBaseTestingUtility}} was deprecated
> back in 2.0.0 and should be removed for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23771) [Flakey Tests] Test TestSplitTransactionOnCluster Again
[ https://issues.apache.org/jira/browse/HBASE-23771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029112#comment-17029112 ] Nick Dimiduk commented on HBASE-23771: -- +1 > [Flakey Tests] Test TestSplitTransactionOnCluster Again > --- > > Key: HBASE-23771 > URL: https://issues.apache.org/jira/browse/HBASE-23771 > Project: HBase > Issue Type: Sub-task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: > 0001-HBASE-23771-Flakey-Tests-Test-TestSplitTransactionOn.patch, Screen Shot > 2020-01-31 at 8.37.13 AM.png > > > Parent fix had the test failures in GCE go from 35% to 4%. Let me see if can > clear the remaining fails. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #1115: HBASE-23782 We still reference the hard coded meta descriptor in some…
saintstack commented on a change in pull request #1115: HBASE-23782 We still
reference the hard coded meta descriptor in some…
URL: https://github.com/apache/hbase/pull/1115#discussion_r374233401
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/TestHBaseMetaEdit.java
##
@@ -63,6 +63,23 @@ public void after() throws Exception {
UTIL.shutdownMiniCluster();
}
+ // make sure that with every possible way, we get the same meta table
descriptor.
+ private TableDescriptor getMetaDescriptor() throws TableNotFoundException,
IOException {
+Admin admin = UTIL.getAdmin();
+TableDescriptor get = admin.getDescriptor(TableName.META_TABLE_NAME);
+TableDescriptor list =
+ admin.listTableDescriptors(true).stream().filter(td ->
td.isMetaTable()).findAny().get();
+TableDescriptor listByName =
+
admin.listTableDescriptors(Collections.singletonList(TableName.META_TABLE_NAME)).get(0);
+TableDescriptor listByNs =
+
admin.listTableDescriptorsByNamespace(NamespaceDescriptor.SYSTEM_NAMESPACE_NAME).stream()
+.filter(td -> td.isMetaTable()).findAny().get();
+assertEquals(get, list);
+assertEquals(get, listByName);
+assertEquals(get, listByNs);
+return get;
+ }
+
Review comment:
Nice test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack merged pull request #1109: HBASE-23780 Edit of test classifications
saintstack merged pull request #1109: HBASE-23780 Edit of test classifications URL: https://github.com/apache/hbase/pull/1109 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1109: HBASE-23780 Edit of test classifications
saintstack commented on issue #1109: HBASE-23780 Edit of test classifications URL: https://github.com/apache/hbase/pull/1109#issuecomment-581520992 Merged. This patch doesn't touch failing test and can't repro locally. Will address in another issue if it persists. Thanks for the reviews all. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 opened a new pull request #1118: HBASE-23276 Add admin methods to get tables within a group
Apache9 opened a new pull request #1118: HBASE-23276 Add admin methods to get tables within a group URL: https://github.com/apache/hbase/pull/1118 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23782) We still reference the hard coded meta descriptor in some places when listing table descriptors
[
https://issues.apache.org/jira/browse/HBASE-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028957#comment-17028957
]
Hudson commented on HBASE-23782:
Results for branch master
[build #1618 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1618/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> We still reference the hard coded meta descriptor in some places when listing
> table descriptors
> ---
>
> Key: HBASE-23782
> URL: https://issues.apache.org/jira/browse/HBASE-23782
> Project: HBase
> Issue Type: Bug
> Components: meta
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23777) Remove deprecated createTableDescriptor(String, int, int, int, KeepDeletedCells) from HBaseTestingUtility
[
https://issues.apache.org/jira/browse/HBASE-23777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028956#comment-17028956
]
Hudson commented on HBASE-23777:
Results for branch master
[build #1618 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1618/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1618//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove deprecated createTableDescriptor(String, int, int, int,
> KeepDeletedCells) from HBaseTestingUtility
> -
>
> Key: HBASE-23777
> URL: https://issues.apache.org/jira/browse/HBASE-23777
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Minor
> Fix For: 3.0.0
>
>
> {{createTableDescriptor(String, int, int, int, KeepDeletedCells)}} in
> {{HBaseTestingUtility}} was deprecated back in 2.0.0 and should be removed
> for 3.0.0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] brfrn169 commented on issue #1114: HBASE-23146 Support CheckAndMutate with multi conditions
brfrn169 commented on issue #1114: HBASE-23146 Support CheckAndMutate with multi conditions URL: https://github.com/apache/hbase/pull/1114#issuecomment-581389004 The test failures in the last QA are not related to the patch. I ran the failed tests locally and they were successful. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1114: HBASE-23146 Support CheckAndMutate with multi conditions
Apache-HBase commented on issue #1114: HBASE-23146 Support CheckAndMutate with multi conditions URL: https://github.com/apache/hbase/pull/1114#issuecomment-581382054 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 8 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 52s | master passed | | +1 :green_heart: | compile | 3m 39s | master passed | | +1 :green_heart: | checkstyle | 3m 13s | master passed | | +1 :green_heart: | shadedjars | 5m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 21s | master passed | | +0 :ok: | spotbugs | 1m 37s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 12m 22s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 36s | the patch passed | | +1 :green_heart: | compile | 3m 43s | the patch passed | | +1 :green_heart: | cc | 3m 43s | the patch passed | | +1 :green_heart: | javac | 3m 43s | the patch passed | | +1 :green_heart: | checkstyle | 0m 9s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 0m 11s | The patch passed checkstyle in hbase-protocol | | +1 :green_heart: | checkstyle | 0m 33s | hbase-client: The patch generated 0 new + 85 unchanged - 26 fixed = 85 total (was 111) | | +1 :green_heart: | checkstyle | 1m 18s | hbase-server: The patch generated 0 new + 449 unchanged - 10 fixed = 449 total (was 459) | | +1 :green_heart: | checkstyle | 0m 41s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 22s | the patch passed | | +1 :green_heart: | javadoc | 2m 23s | the patch passed | | +1 :green_heart: | findbugs | 13m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 0m 27s | hbase-protocol in the patch passed. | | +1 :green_heart: | unit | 2m 3s | hbase-client in the patch passed. | | -1 :x: | unit | 161m 27s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 3m 47s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 5m 12s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 2m 36s | The patch does not generate ASF License warnings. | | | | 273m 14s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.quotas.TestClusterScopeQuotaThrottle | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1114 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux d30055d5e41c 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1114/out/precommit/personality/provided.sh | | git revision | master / ab0039974d | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1114/2/testReport/ | | Max. process+thread count | 4821
[jira] [Updated] (HBASE-23235) Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion
[ https://issues.apache.org/jira/browse/HBASE-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23235: -- Fix Version/s: HBASE-22514 > Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion > -- > > Key: HBASE-23235 > URL: https://issues.apache.org/jira/browse/HBASE-23235 > Project: HBase > Issue Type: Sub-task > Components: rsgroup, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-22514 > > > It is not allowed to change the table descriptor for system tables, > especially that the table descriptor for meta table is hard coded. We need to > find a way to deal with this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23235) Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion
[ https://issues.apache.org/jira/browse/HBASE-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-23235. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to branch HBASE-22514. Thanks [~zghao] for reviewing. > Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion > -- > > Key: HBASE-23235 > URL: https://issues.apache.org/jira/browse/HBASE-23235 > Project: HBase > Issue Type: Sub-task > Components: rsgroup, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: HBASE-22514 > > > It is not allowed to change the table descriptor for system tables, > especially that the table descriptor for meta table is hard coded. We need to > find a way to deal with this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23235) Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion
[ https://issues.apache.org/jira/browse/HBASE-23235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-23235: -- Component/s: test rsgroup > Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion > -- > > Key: HBASE-23235 > URL: https://issues.apache.org/jira/browse/HBASE-23235 > Project: HBase > Issue Type: Sub-task > Components: rsgroup, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > It is not allowed to change the table descriptor for system tables, > especially that the table descriptor for meta table is hard coded. We need to > find a way to deal with this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-22813) Revisit the logic of when we can remove the last server from a rsgroup and when can we remove a rsgroup
[ https://issues.apache.org/jira/browse/HBASE-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-22813. --- Resolution: Not A Problem > Revisit the logic of when we can remove the last server from a rsgroup and > when can we remove a rsgroup > --- > > Key: HBASE-22813 > URL: https://issues.apache.org/jira/browse/HBASE-22813 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Priority: Major > > The logic is a bit confusing for now. We do not allow creating tables to an > empty rs group, but it is fine to remove all the regionservers from a > rsgroup. And we will even automatically remove region servers when > clearDeadServers... -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #1117: HBASE-23235 Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion
Apache9 merged pull request #1117: HBASE-23235 Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion URL: https://github.com/apache/hbase/pull/1117 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1117: HBASE-23235 Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion
Apache-HBase commented on issue #1117: HBASE-23235 Re-enable TestRSGroupsKillRS.testLowerMetaGroupVersion URL: https://github.com/apache/hbase/pull/1117#issuecomment-581341717 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 30s | HBASE-22514 passed | | +1 :green_heart: | compile | 0m 57s | HBASE-22514 passed | | +1 :green_heart: | checkstyle | 1m 9s | HBASE-22514 passed | | +1 :green_heart: | shadedjars | 4m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | HBASE-22514 passed | | +0 :ok: | spotbugs | 4m 51s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 50s | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 47s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | | +1 :green_heart: | findbugs | 4m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 54s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 205m 5s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1117/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1117 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ae1f9a5893d5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1117/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 86defbf835 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1117/1/testReport/ | | Max. process+thread count | 4882 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1117/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23742) Mob Data, threshold crossed data is considering as normal data.
[ https://issues.apache.org/jira/browse/HBASE-23742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17028839#comment-17028839 ] Anoop Sam John commented on HBASE-23742: Agree with Pankaj. Better not to add more complexity in this area. Anyway nice catch. We now know there is possibility of MOB data temp in Hfiles only when the WAL split + replay happens. Thanks [~sreenivasulureddy] for the tests. > Mob Data, threshold crossed data is considering as normal data. > --- > > Key: HBASE-23742 > URL: https://issues.apache.org/jira/browse/HBASE-23742 > Project: HBase > Issue Type: Sub-task > Components: MTTR >Affects Versions: 3.0.0, 2.3.0 >Reporter: Y. SREENIVASULU REDDY >Priority: Minor > > Steps to reproduce this issue. > 1. create a table with 1 region, and mob enabled, keep threshold value to 5. > 2. Load data into the table, keep the value size should be more than 5. > 3. flush the table. > 4. observe the mobdir and data dir, hfiles should be there. > 5. load data again with different data set, keep the value size is greater > than 5. > 6. Kill -9 RS where table region is online > 7. Start RS > check the mob dir and data dir, both should have 2 hfiles each. > But data dir only have 2 hfiles, that means mob threshold crossed data is > considered as normal data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
