[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966441#comment-16966441
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #169 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/169/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/169//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/169//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/169//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore
virajjasani commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore URL: https://github.com/apache/hbase/pull/773#issuecomment-549237410 Please review @xcangCRM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] binlijin commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore
binlijin commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore URL: https://github.com/apache/hbase/pull/773#issuecomment-549235941 Need other committer to approve it except me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23245) All MutableHistogram implementations should remove maxExpected
[ https://issues.apache.org/jira/browse/HBASE-23245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966419#comment-16966419 ] Viraj Jasani commented on HBASE-23245: -- Please review [~enis] > All MutableHistogram implementations should remove maxExpected > -- > > Key: HBASE-23245 > URL: https://issues.apache.org/jira/browse/HBASE-23245 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 3.0.0, 2.3.0, 1.6.0 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 2.3.0, 1.6.0 > > > MutableHistogram is using HistogramImpl() and not using maxExpected. From all > implementation classes, maxExpected should be removed as constructor > argument. Also, HistogramImpl() should pass long casting for > Integer.MAX_VALUE << 2 else it is treated as -4 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] binlijin commented on a change in pull request #782: HBASE-23241 TestExecutorService sometimes fail
binlijin commented on a change in pull request #782: HBASE-23241
TestExecutorService sometimes fail
URL: https://github.com/apache/hbase/pull/782#discussion_r341899242
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/executor/TestExecutorService.java
##
@@ -231,9 +231,11 @@ public void testSnapshotHandlers() throws Exception {
executorService.startExecutorService(ExecutorType.MASTER_SNAPSHOT_OPERATIONS,
1);
CountDownLatch latch = new CountDownLatch(1);
+final AtomicInteger counter = new AtomicInteger(0);
Review comment:
Thanks for you advice
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani removed a comment on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore
virajjasani removed a comment on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore URL: https://github.com/apache/hbase/pull/773#issuecomment-549007166 Done @binlijin Please take a look This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore
virajjasani commented on issue #773: HBASE-23212 : Dynamically reload configs for Region Recovery chore URL: https://github.com/apache/hbase/pull/773#issuecomment-549218903 Please review @binlijin This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog…
virajjasani commented on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog… URL: https://github.com/apache/hbase/pull/787#issuecomment-549218778 Please review @enis @elliottneilclark This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani removed a comment on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog…
virajjasani removed a comment on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog… URL: https://github.com/apache/hbase/pull/787#issuecomment-549109204 Please review @enis @elliottneilclark This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r341898721 ## File path: hbase-shell/src/main/ruby/shell.rb ## @@ -333,10 +333,12 @@ def help_footer normalizer_switch normalizer_enabled is_in_maintenance_mode +clear_slow_responses Review comment: @guangxuCheng I have made changes. Please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] binlijin commented on issue #782: HBASE-23241 TestExecutorService sometimes fail
binlijin commented on issue #782: HBASE-23241 TestExecutorService sometimes fail URL: https://github.com/apache/hbase/pull/782#issuecomment-549217619 @wchevreuil it is depends on the thread schedule, so how to add a test to the flaky list? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bsglz edited a comment on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref
bsglz edited a comment on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref URL: https://github.com/apache/hbase/pull/764#issuecomment-549212614 > That's good. Would u mind collecting some heap usage/GC stats, together with overall RS metrics (such as write/read requests, compactions, etc) from few days period? I have added a doc for it as a attachment of issue,link at below： https://issues.apache.org/jira/secure/attachment/12984751/HBASE-23223-RS%20metrics%20with%20patch.pdf This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bsglz edited a comment on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref
bsglz edited a comment on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref URL: https://github.com/apache/hbase/pull/764#issuecomment-549212614 > That's good. Would u mind collecting some heap usage/GC stats, together with overall RS metrics (such as write/read requests, compactions, etc) from few days period? I has added a doc for it as a attachment of issue,link at below： https://issues.apache.org/jira/secure/attachment/12984751/HBASE-23223-RS%20metrics%20with%20patch.pdf This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] bsglz commented on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref
bsglz commented on issue #764: HBASE-23223 Support the offsetLock of bucketCache to use strong ref URL: https://github.com/apache/hbase/pull/764#issuecomment-549212614 > That's good. Would u mind collecting some heap usage/GC stats, together with overall RS metrics (such as write/read requests, compactions, etc) from few days period? I has added a doc for it as a attachment of issue,link at below： https://issues.apache.org/jira/secure/attachment/12984751/HBASE-23223-RS%20metrics%20with%20patch.pdf This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23223) Support the offsetLock of bucketCache to use strong reference
[ https://issues.apache.org/jira/browse/HBASE-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Wang updated HBASE-23223: --- Attachment: HBASE-23223-RS metrics with patch.pdf > Support the offsetLock of bucketCache to use strong reference > -- > > Key: HBASE-23223 > URL: https://issues.apache.org/jira/browse/HBASE-23223 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: HBASE-23223-RS metrics with patch.pdf, > HBASE-23223-benchmark.pdf > > > My live cluster env config below: > hbase version: cdh6.0.1(apache hbase2.0.0) > hbase config: bucketCache(70g),blocksize(16k) > java version: 1.8.0_51 > java config: heap(32g),-XX:+UseG1GC -XX:MaxGCPauseMillis=100 > -XX:+ParallelRefProcEnabled > > The offsetLock of BucketCache use SoftReference for its > ReentrantReadWriteLock,when bucketCache is big,it will cost too much time in > Gc ref-proc. > eg : > 2019-09-29T01:55:45.186+0800: 365222.053: > [GC remark > 2019-09-29T01:55:45.186+0800: 365222.053: > [Finalize Marking, 0.0016327 secs] > 2019-09-29T01:55:45.188+0800: 365222.054: > [GC ref-proc > 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 1264586 refs, > 0.3151392 secs] > 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317 refs, > 0.0024381 secs] > 2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference, 9791 refs, > 0.0037445 secs] > 2019-09-29T01:55:45.509+0800: 365222.376: [PhantomReference, 0 refs, 1963 > refs, 0.0018941 secs] > 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 0.0001156 > secs] > , 1.4554361 secs] > 2019-09-29T01:55:46.643+0800: 365223.510: > [Unloading, 0.0211370 secs] > , 1.4851728 secs] > > I think the strong reference is a better choice in some case ,so i changed > it and get good effect on my live cluster: > > 2019-10-26T02:29:10.248+0800: 131255.447: > [GC remark > 2019-10-26T02:29:10.248+0800: 131255.447: > [Finalize Marking, 0.0011426 secs] > 2019-10-26T02:29:10.249+0800: 131255.448: > [GC ref-proc > 2019-10-26T02:29:10.249+0800: 131255.448: [SoftReference, 213 refs, > 0.0002216 secs] > 2019-10-26T02:29:10.249+0800: 131255.448: [WeakReference, 2423 refs, > 0.0007833 secs] > 2019-10-26T02:29:10.250+0800: 131255.449: [FinalReference, 4991 refs, > 0.0061449 secs] > 2019-10-26T02:29:10.256+0800: 131255.455: [PhantomReference, 0 refs, 7940 > refs, 0.0011711 secs] > 2019-10-26T02:29:10.257+0800: 131255.456: [JNI Weak Reference, 0.579 > secs] > , 0.0086389 secs] > 2019-10-26T02:29:10.257+0800: 131255.456: > [Unloading, 0.0123417 secs] > , 0.0294484 secs] > > Strong reference will always hold the mem,so it maybe not fit for some case > that have not enough mem, so i make it configable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] guangxuCheng commented on a change in pull request #754: HBASE-22978 : Online slow response log
guangxuCheng commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r341889912 ## File path: hbase-shell/src/main/ruby/shell.rb ## @@ -333,10 +333,12 @@ def help_footer normalizer_switch normalizer_enabled is_in_maintenance_mode +clear_slow_responses Review comment: I think it is more readable if we keep the shell command and method name consistent. @apurtell WDYT？ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23223) Support the offsetLock of bucketCache to use strong reference
[ https://issues.apache.org/jira/browse/HBASE-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966350#comment-16966350 ] Zheng Wang commented on HBASE-23223: In the strong ref test,it didn't occur full gc. Compared with soft ref,the strong ref doesn't need a wrapper class,so it cost lesser mem. > Support the offsetLock of bucketCache to use strong reference > -- > > Key: HBASE-23223 > URL: https://issues.apache.org/jira/browse/HBASE-23223 > Project: HBase > Issue Type: Improvement > Components: BucketCache >Reporter: Zheng Wang >Assignee: Zheng Wang >Priority: Major > Attachments: HBASE-23223-benchmark.pdf > > > My live cluster env config below: > hbase version: cdh6.0.1(apache hbase2.0.0) > hbase config: bucketCache(70g),blocksize(16k) > java version: 1.8.0_51 > java config: heap(32g),-XX:+UseG1GC -XX:MaxGCPauseMillis=100 > -XX:+ParallelRefProcEnabled > > The offsetLock of BucketCache use SoftReference for its > ReentrantReadWriteLock,when bucketCache is big,it will cost too much time in > Gc ref-proc. > eg : > 2019-09-29T01:55:45.186+0800: 365222.053: > [GC remark > 2019-09-29T01:55:45.186+0800: 365222.053: > [Finalize Marking, 0.0016327 secs] > 2019-09-29T01:55:45.188+0800: 365222.054: > [GC ref-proc > 2019-09-29T01:55:45.188+0800: 365222.054: [SoftReference, 1264586 refs, > 0.3151392 secs] > 2019-09-29T01:55:45.503+0800: 365222.370: [WeakReference, 4317 refs, > 0.0024381 secs] > 2019-09-29T01:55:45.505+0800: 365222.372: [FinalReference, 9791 refs, > 0.0037445 secs] > 2019-09-29T01:55:45.509+0800: 365222.376: [PhantomReference, 0 refs, 1963 > refs, 0.0018941 secs] > 2019-09-29T01:55:45.511+0800: 365222.378: [JNI Weak Reference, 0.0001156 > secs] > , 1.4554361 secs] > 2019-09-29T01:55:46.643+0800: 365223.510: > [Unloading, 0.0211370 secs] > , 1.4851728 secs] > > I think the strong reference is a better choice in some case ,so i changed > it and get good effect on my live cluster: > > 2019-10-26T02:29:10.248+0800: 131255.447: > [GC remark > 2019-10-26T02:29:10.248+0800: 131255.447: > [Finalize Marking, 0.0011426 secs] > 2019-10-26T02:29:10.249+0800: 131255.448: > [GC ref-proc > 2019-10-26T02:29:10.249+0800: 131255.448: [SoftReference, 213 refs, > 0.0002216 secs] > 2019-10-26T02:29:10.249+0800: 131255.448: [WeakReference, 2423 refs, > 0.0007833 secs] > 2019-10-26T02:29:10.250+0800: 131255.449: [FinalReference, 4991 refs, > 0.0061449 secs] > 2019-10-26T02:29:10.256+0800: 131255.455: [PhantomReference, 0 refs, 7940 > refs, 0.0011711 secs] > 2019-10-26T02:29:10.257+0800: 131255.456: [JNI Weak Reference, 0.579 > secs] > , 0.0086389 secs] > 2019-10-26T02:29:10.257+0800: 131255.456: > [Unloading, 0.0123417 secs] > , 0.0294484 secs] > > Strong reference will always hold the mem,so it maybe not fit for some case > that have not enough mem, so i make it configable. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
Apache9 commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549203676 > On rewrite, yeah... we are hosed when Wals are corrupted. There is also fact that this is yet another wal impl apart from one writing data wals It is not easy to use the wal for writing data as it is in the hbase-server module... But anyway, the current implementation of proc wal is too complicated... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log
Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-549192810 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :blue_heart: | prototool | 0m 0s | prototool was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 37s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 17s | master passed | | :green_heart: | compile | 3m 0s | master passed | | :green_heart: | checkstyle | 2m 42s | master passed | | :blue_heart: | refguide | 5m 29s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 5m 1s | master passed | | :blue_heart: | spotbugs | 1m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 23m 15s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 58s | the patch passed | | :green_heart: | compile | 3m 7s | the patch passed | | :green_heart: | cc | 3m 7s | the patch passed | | :green_heart: | javac | 3m 7s | the patch passed | | :green_heart: | checkstyle | 2m 47s | root: The patch generated 0 new + 488 unchanged - 1 fixed = 488 total (was 489) | | :broken_heart: | rubocop | 0m 18s | The patch generated 19 new + 490 unchanged - 0 fixed = 509 total (was 490) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | :blue_heart: | refguide | 5m 50s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | :green_heart: | shadedjars | 4m 44s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 44s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | hbaseprotoc | 8m 35s | the patch passed | | :green_heart: | javadoc | 5m 9s | the patch passed | | :green_heart: | findbugs | 24m 11s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 212m 22s | root in the patch failed. | | :green_heart: | asflicense | 4m 16s | The patch does not generate ASF License warnings. | | | | 350m 53s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestVerifyReplication | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/754 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc prototool rubocop | | uname | Linux 4a215c6d161b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-754/out/precommit/personality/provided.sh | | git revision | master / 3ccfd50bd9 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/artifact/out/branch-site/book.html | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/artifact/out/diff-patch-rubocop.txt | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/3/testReport/ | | Max. process+thread count | 5542 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-common hbase-client hbase-server
[GitHub] [hbase-connectors] busbey commented on issue #46: Bump checkstyle from 8.11 to 8.18
busbey commented on issue #46: Bump checkstyle from 8.11 to 8.18 URL: https://github.com/apache/hbase-connectors/pull/46#issuecomment-549190485 The ci run didn't do checkstyle so please check it before merging This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] busbey commented on issue #43: Bump checkstyle from 8.11 to 8.18
busbey commented on issue #43: Bump checkstyle from 8.11 to 8.18 URL: https://github.com/apache/hbase-operator-tools/pull/43#issuecomment-549187953 We should make sure checkstyle with the change works. Since no Java files or checkstyle configs changed we didn't run it in precommit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] saintstack commented on issue #46: Bump checkstyle from 8.11 to 8.18
saintstack commented on issue #46: Bump checkstyle from 8.11 to 8.18 URL: https://github.com/apache/hbase-connectors/pull/46#issuecomment-549184304 Go ahead and merge if looks good to you Jan? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
saintstack commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549183890 On rewrite, yeah... we are hosed when Wals are corrupted. There is also fact that this is yet another wal impl apart from one writing data wals This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
saintstack commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549183744 Yeah. We have an assert but not on in prod and this is just a bit of debug to make sure which pid is npe This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] HorizonNet commented on issue #43: Bump checkstyle from 8.11 to 8.18
HorizonNet commented on issue #43: Bump checkstyle from 8.11 to 8.18 URL: https://github.com/apache/hbase-operator-tools/pull/43#issuecomment-549181043 How are we handling PRs like this? For me it looks good, but not sure how to proceed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] HorizonNet commented on issue #46: Bump checkstyle from 8.11 to 8.18
HorizonNet commented on issue #46: Bump checkstyle from 8.11 to 8.18 URL: https://github.com/apache/hbase-connectors/pull/46#issuecomment-549181075 How are we handling PRs like this? For me it looks good, but not sure how to proceed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23185) High cpu usage because getTable()#put() gets config value every time
[
https://issues.apache.org/jira/browse/HBASE-23185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966253#comment-16966253
]
Hudson commented on HBASE-23185:
Results for branch branch-1.4
[build #1076 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1076/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1076//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1076//JDK7_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1076//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> High cpu usage because getTable()#put() gets config value every time
>
>
> Key: HBASE-23185
> URL: https://issues.apache.org/jira/browse/HBASE-23185
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 1.5.0, 1.4.10, 1.2.12, 1.3.5
>Reporter: Shinya Yoshida
>Assignee: Shinya Yoshida
>Priority: Major
> Labels: performance
> Fix For: 1.6.0
>
> Attachments: Screenshot from 2019-10-18 12-38-14.png, Screenshot from
> 2019-10-18 13-03-24.png
>
>
> When we analyzed the performance of our hbase application with many puts, we
> found that Configuration methods use many CPU resources:
> !Screenshot from 2019-10-18 12-38-14.png|width=460,height=205!
> As you can see, getTable().put() is calling Configuration methods which cause
> regex or synchronization by Hashtable.
> This should not happen in 0.99.2 because
> https://issues.apache.org/jira/browse/HBASE-12128 addressed such an issue.
> However, it's reproducing nowadays by bugs or leakages after many code
> evoluations between 0.9x and 1.x.
> #
> [https://github.com/apache/hbase/blob/dd9eadb00f9dcd071a246482a11dfc7d63845f00/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java#L369-L374]
> ** finishSetup is called every new HTable() e.g. every con.getTable()
> ** So getInt is called everytime and it does regex
> #
> [https://github.com/apache/hbase/blob/dd9eadb00f9dcd071a246482a11dfc7d63845f00/hbase-client/src/main/java/org/apache/hadoop/hbase/client/BufferedMutatorImpl.java#L115]
> ** BufferedMutatorImpl is created every first put for HTable e.g.
> con.getTable().put()
> ** Create ConnectionConf every time in BufferedMutatorImpl constructor
> ** ConnectionConf gets config value in the constructor
> #
> [https://github.com/apache/hbase/blob/dd9eadb00f9dcd071a246482a11dfc7d63845f00/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java#L326]
> ** AsyncProcess is created in BufferedMutatorImpl constructor, so new
> AsyncProcess is created by con.getTable().put()
> ** AsyncProcess parse many configurations
> So, con.getTable().put() is heavy operation for CPU because of getting config
> value.
>
> With in-house patch for this issue, we observed about 10% improvement on
> max-throughput (e.g. CPU usage) at client-side:
> !Screenshot from 2019-10-18 13-03-24.png|width=508,height=223!
>
> Seems branch-2 is not affected because client implementation has been changed
> dramatically.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341868947
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HMasterAsyncRegistry.java
##
@@ -0,0 +1,230 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.MetaRegionsNotAvailableException;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsResponse;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.hadoop.hbase.HConstants.DEFAULT_HBASE_RPC_TIMEOUT;
+import static org.apache.hadoop.hbase.HConstants.HBASE_RPC_TIMEOUT_KEY;
+
+
+/**
+ * Fetches the meta information directly from HMaster by making relevant RPCs.
HMaster RPC end points are looked up
+ * via client configuration 'hbase.client.asyncregistry.masteraddrs' as comma
separated list of :
+ * values. Should be set in conjunction with 'hbase.client.registry.impl' set
to this class impl.
+ *
+ * The class does not cache anything. It is the responsibility of the callers
to cache and avoid repeated requests.
+ */
+@InterfaceAudience.Private
+public class HMasterAsyncRegistry implements AsyncRegistry {
+private static final Logger LOG =
LoggerFactory.getLogger(ZKAsyncRegistry.class);
+public static final String CONF_KEY =
"hbase.client.asyncregistry.masteraddrs";
+public static final String DEFAULT_HOST_PORT = "localhost:" +
HConstants.DEFAULT_MASTER_PORT;
Review comment:
This can also be private
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341869087
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaRegionLocationCache.java
##
@@ -0,0 +1,167 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.master;
+
+import org.apache.hadoop.hbase.MetaRegionsNotAvailableException;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.types.CopyOnWriteArrayMap;
+import org.apache.hadoop.hbase.zookeeper.MetaTableLocator;
+import org.apache.hadoop.hbase.zookeeper.ZKListener;
+import org.apache.hadoop.hbase.zookeeper.ZKWatcher;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.KeeperException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.ConcurrentNavigableMap;
+
+/**
+ * A cache of meta region location metadata. This cache is used to serve
'GetMetaLocations' RPCs from clients. Registers
+ * a listener on ZK to track changes to the meta table znodes. Clients are
expected to retry if the meta
+ * information is stale. This class is thread-safe.
+ */
+@InterfaceAudience.Private
+public class MetaRegionLocationCache extends ZKListener {
+
+private static final Logger LOG =
LoggerFactory.getLogger(MetaRegionLocationCache.class);
+
+// Maximum number of times we retry when ZK operation times out. Should
this be configurable?
+private static final int MAX_ZK_META_FETCH_RETRIES = 10;
Review comment:
I think better to make it configurable and may be keep 10(not sure about
right value) as default in the absence of site config.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341868902
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/HMasterAsyncRegistry.java
##
@@ -0,0 +1,230 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.HRegionLocation;
+import org.apache.hadoop.hbase.MasterNotRunningException;
+import org.apache.hadoop.hbase.MetaRegionsNotAvailableException;
+import org.apache.hadoop.hbase.RegionLocations;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.ipc.HBaseRpcController;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.ipc.RpcControllerFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsResponse;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.TimeUnit;
+
+import static org.apache.hadoop.hbase.HConstants.DEFAULT_HBASE_RPC_TIMEOUT;
+import static org.apache.hadoop.hbase.HConstants.HBASE_RPC_TIMEOUT_KEY;
+
+
+/**
+ * Fetches the meta information directly from HMaster by making relevant RPCs.
HMaster RPC end points are looked up
+ * via client configuration 'hbase.client.asyncregistry.masteraddrs' as comma
separated list of :
+ * values. Should be set in conjunction with 'hbase.client.registry.impl' set
to this class impl.
+ *
+ * The class does not cache anything. It is the responsibility of the callers
to cache and avoid repeated requests.
+ */
+@InterfaceAudience.Private
+public class HMasterAsyncRegistry implements AsyncRegistry {
+private static final Logger LOG =
LoggerFactory.getLogger(ZKAsyncRegistry.class);
+public static final String CONF_KEY =
"hbase.client.asyncregistry.masteraddrs";
Review comment:
Same here, good to keep it private unless source code needs it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341868745
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterAsyncRegistryRPCs.java
##
@@ -0,0 +1,201 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.zookeeper.ZKWatcher;
+import org.apache.hadoop.hbase.zookeeper.ZNodePaths;
+import org.apache.hbase.thirdparty.com.google.protobuf.BlockingRpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetPortInfoRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetPortInfoResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveResponse;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+@Category({ MasterTests.class})
+public class TestHMasterAsyncRegistryRPCs {
+private static final Logger LOG =
LoggerFactory.getLogger(TestHMasterAsyncRegistryRPCs.class);
+
+private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+private static RpcClient rpcClient;
+private static HMaster activeMaster;
+private static List standByMasters;
+
+@BeforeClass
+public static void setUp() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.set(HConstants.MASTER_PORT, "0");
+conf.setStrings(HConstants.ZOOKEEPER_ZNODE_PARENT, "/metacachetest");
+TEST_UTIL.startMiniCluster(2);
+activeMaster = TEST_UTIL.getHBaseCluster().getMaster();
+standByMasters = new ArrayList<>();
+// Create a few standby masters.
+for (int i = 0; i < 2; i++) {
+
standByMasters.add(TEST_UTIL.getHBaseCluster().startMaster().getMaster());
+}
+rpcClient = RpcClientFactory.createClient(conf,
HConstants.CLUSTER_ID_DEFAULT);
+}
+
+@AfterClass
+public static void tearDown() throws Exception {
+if (rpcClient != null) {
+rpcClient.close();
+}
+TEST_UTIL.shutdownMiniCluster();
+}
+
+/**
+ * Verifies that getMetaReplicaIdFromPath() parses the full meta znode
paths correctly.
+ * @throws IOException
+ */
+@Test
+public void TestGetMetaReplicaIdFromPath() throws IOException {
+ZKWatcher zk = TEST_UTIL.getZooKeeperWatcher();
+String metaZnode= ZNodePaths.joinZNode(zk.getZNodePaths().baseZNode,
zk.getZNodePaths().metaZNodePrefix);
+assertEquals(0,
zk.getZNodePaths().getMetaReplicaIdFromPath(metaZnode));
+for (int i = 1; i <
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341869315
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestHMasterAsyncRegistryRPCs.java
##
@@ -0,0 +1,201 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.ServerName;
+import org.apache.hadoop.hbase.ipc.RpcClient;
+import org.apache.hadoop.hbase.ipc.RpcClientFactory;
+import org.apache.hadoop.hbase.security.User;
+import org.apache.hadoop.hbase.testclassification.MasterTests;
+import org.apache.hadoop.hbase.zookeeper.ZKWatcher;
+import org.apache.hadoop.hbase.zookeeper.ZNodePaths;
+import org.apache.hbase.thirdparty.com.google.protobuf.BlockingRpcChannel;
+import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException;
+import org.junit.After;
+import org.junit.AfterClass;
+import org.junit.Before;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetPortInfoRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetPortInfoResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsResponse;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveRequest;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveResponse;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+@Category({ MasterTests.class})
+public class TestHMasterAsyncRegistryRPCs {
+private static final Logger LOG =
LoggerFactory.getLogger(TestHMasterAsyncRegistryRPCs.class);
+
+private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+private static RpcClient rpcClient;
+private static HMaster activeMaster;
+private static List standByMasters;
+
+@BeforeClass
+public static void setUp() throws Exception {
+Configuration conf = TEST_UTIL.getConfiguration();
+conf.set(HConstants.MASTER_PORT, "0");
+conf.setStrings(HConstants.ZOOKEEPER_ZNODE_PARENT, "/metacachetest");
+TEST_UTIL.startMiniCluster(2);
+activeMaster = TEST_UTIL.getHBaseCluster().getMaster();
+standByMasters = new ArrayList<>();
+// Create a few standby masters.
+for (int i = 0; i < 2; i++) {
+
standByMasters.add(TEST_UTIL.getHBaseCluster().startMaster().getMaster());
+}
+rpcClient = RpcClientFactory.createClient(conf,
HConstants.CLUSTER_ID_DEFAULT);
+}
+
+@AfterClass
+public static void tearDown() throws Exception {
+if (rpcClient != null) {
+rpcClient.close();
+}
+TEST_UTIL.shutdownMiniCluster();
+}
+
+/**
+ * Verifies that getMetaReplicaIdFromPath() parses the full meta znode
paths correctly.
+ * @throws IOException
+ */
+@Test
+public void TestGetMetaReplicaIdFromPath() throws IOException {
+ZKWatcher zk = TEST_UTIL.getZooKeeperWatcher();
+String metaZnode= ZNodePaths.joinZNode(zk.getZNodePaths().baseZNode,
zk.getZNodePaths().metaZNodePrefix);
+assertEquals(0,
zk.getZNodePaths().getMetaReplicaIdFromPath(metaZnode));
+for (int i = 1; i <
[GitHub] [hbase] virajjasani commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
virajjasani commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341868848
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegistryFactory.java
##
@@ -27,7 +27,7 @@
@InterfaceAudience.Private
final class AsyncRegistryFactory {
- static final String REGISTRY_IMPL_CONF_KEY = "hbase.client.registry.impl";
+ public static final String REGISTRY_IMPL_CONF_KEY =
"hbase.client.registry.impl";
Review comment:
IMHO unless some source code requires this to be public, we can keep this
unchanged. For test case, we can directly use "hbase.client.registry.impl".
Should be fine?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r341606537 ## File path: hbase-shell/src/main/ruby/shell.rb ## @@ -333,10 +333,12 @@ def help_footer normalizer_switch normalizer_enabled is_in_maintenance_mode +clear_slow_responses Review comment: Using clear_slow_responses and get_slow_responses might be simpler and I tried following example mentioned by @apurtell on Jira: https://issues.apache.org/jira/browse/HBASE-22978 However, since JAVA code references refer to "log", I believe it is fine to update here also. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r341837096 ## File path: hbase-shell/src/main/ruby/shell.rb ## @@ -333,10 +333,12 @@ def help_footer normalizer_switch normalizer_enabled is_in_maintenance_mode +clear_slow_responses Review comment: Is this fine with you @guangxuCheng ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation URL: https://github.com/apache/hbase/pull/781#discussion_r341853924 ## File path: hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ## @@ -213,6 +213,10 @@ private volatile AsyncClusterConnection asyncConnection; + // Tracks any other connections created with custom client config. Used for testing clients with custom + // configurations. Tracked here so that they can be cleaned up on close() / restart. + private List customConnections = Collections.synchronizedList(new ArrayList<>()); + Review comment: I didn't quite follow why you needed to define these extra connections. Are you using both types of registry implementation on a single util instance? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
wchevreuil commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341853531
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java
##
@@ -3188,6 +3188,7 @@ public static String
toLockJson(List lockedRes
}
public static HBaseProtos.RegionLocation toRegionLocation(HRegionLocation
loc) {
+if (loc == null) return null;
Review comment:
just curious, did you face NPE conditions while testing this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation URL: https://github.com/apache/hbase/pull/781#discussion_r341846132 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncRegistry.java ## @@ -45,21 +44,11 @@ */ CompletableFuture getClusterId(); - /** Review comment: So no need to have _getCurrentNrHRS_ and _getMasterInfoPort_ methods in either ZK or Master based registry anymore? No client usage for this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
wchevreuil commented on a change in pull request #781: HBASE-18095:
Zookeeper-less client connection implementation
URL: https://github.com/apache/hbase/pull/781#discussion_r341854149
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHMasterAsyncRegistry.java
##
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.client;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseTestingUtility;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.master.HMaster;
+import org.apache.hadoop.hbase.testclassification.ClientTests;
+import org.apache.hadoop.hbase.testclassification.MediumTests;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.yarn.webapp.hamlet.Hamlet;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.net.HostAndPort;
+
+import static junit.framework.TestCase.assertTrue;
+import static org.apache.hadoop.hbase.HConstants.META_REPLICAS_NUM;
+import static org.junit.Assert.assertEquals;
+
+@Category({ MediumTests.class, ClientTests.class })
+public class TestHMasterAsyncRegistry {
+private static final Logger LOG =
LoggerFactory.getLogger(TestHMasterAsyncRegistry.class);
+private static final HBaseTestingUtility TEST_UTIL = new
HBaseTestingUtility();
+private static final TableName TEST_TABLE = TableName.valueOf("foo");
+private static final byte[] COL_FAM = Bytes.toBytes("cf");
+private static final byte[] QUALIFIER = Bytes.toBytes("col");
+
+
+// Automatically cleaned up on cluster shutdown.
+private static AsyncClusterConnection customConnection;
+private static HMasterAsyncRegistry REGISTRY;
+
+@BeforeClass
+public static void setUp() throws Exception {
+TEST_UTIL.getConfiguration().setInt(META_REPLICAS_NUM, 3);
+TEST_UTIL.startMiniCluster(3);
+Configuration conf = new Configuration(TEST_UTIL.getConfiguration());
+// Set the master host:port in the client config used to create a
custom connection.
+String masterHostName =
TEST_UTIL.getMiniHBaseCluster().getMaster().getServerName().getHostname();
+int masterPort =
TEST_UTIL.getMiniHBaseCluster().getMaster().getServerName().getPort();
+conf.set(HMasterAsyncRegistry.CONF_KEY,
HostAndPort.fromParts(masterHostName, masterPort).toString());
+conf.set(AsyncRegistryFactory.REGISTRY_IMPL_CONF_KEY,
HMasterAsyncRegistry.class.getName());
Review comment:
Could be set on _TEST_UTIL_ config before starting the cluster, so that its
async connection impl already uses _HMasterAsyncRegistry_?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation
wchevreuil commented on a change in pull request #781: HBASE-18095: Zookeeper-less client connection implementation URL: https://github.com/apache/hbase/pull/781#discussion_r341846499 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/HMasterAsyncRegistry.java ## @@ -0,0 +1,230 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.client; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.HRegionLocation; +import org.apache.hadoop.hbase.MasterNotRunningException; +import org.apache.hadoop.hbase.MetaRegionsNotAvailableException; +import org.apache.hadoop.hbase.RegionLocations; +import org.apache.hadoop.hbase.ServerName; +import org.apache.hadoop.hbase.ipc.HBaseRpcController; +import org.apache.hadoop.hbase.ipc.RpcClient; +import org.apache.hadoop.hbase.ipc.RpcClientFactory; +import org.apache.hadoop.hbase.ipc.RpcControllerFactory; +import org.apache.hadoop.hbase.security.User; +import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil; +import org.apache.hbase.thirdparty.com.google.common.base.Preconditions; +import org.apache.hbase.thirdparty.com.google.protobuf.ServiceException; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.MasterService; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveRequest; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.IsActiveResponse; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdRequest; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetClusterIdResponse; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsRequest; +import org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos.GetMetaLocationsResponse; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; + +import static org.apache.hadoop.hbase.HConstants.DEFAULT_HBASE_RPC_TIMEOUT; +import static org.apache.hadoop.hbase.HConstants.HBASE_RPC_TIMEOUT_KEY; + + +/** + * Fetches the meta information directly from HMaster by making relevant RPCs. HMaster RPC end points are looked up + * via client configuration 'hbase.client.asyncregistry.masteraddrs' as comma separated list of : + * values. Should be set in conjunction with 'hbase.client.registry.impl' set to this class impl. + * + * The class does not cache anything. It is the responsibility of the callers to cache and avoid repeated requests. Review comment: I think this is fine. RPC requests to master might be heavier than querying ZK, but as this is targeted as private, will be used mostly internally, and I believe existing callees such as ConnectionUtils already implement caching logic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23244) NPEs running Canary
[
https://issues.apache.org/jira/browse/HBASE-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965672#comment-16965672
]
Hudson commented on HBASE-23244:
Results for branch master
[build #1525 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1525/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1525//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1525//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1525//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> NPEs running Canary
> ---
>
> Key: HBASE-23244
> URL: https://issues.apache.org/jira/browse/HBASE-23244
> Project: HBase
> Issue Type: Bug
> Components: canary
>Affects Versions: 2.2.2
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3
>
>
> There's a few. Here is one:
> {code}
> 19/11/01 23:54:16 ERROR tool.Canary: Run regionMonitor failed
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.tool.CanaryTool.sniff(CanaryTool.java:1582)
> at org.apache.hadoop.hbase.tool.CanaryTool.sniff(CanaryTool.java:1562)
> at
> org.apache.hadoop.hbase.tool.CanaryTool.access$100(CanaryTool.java:124)
> at
> org.apache.hadoop.hbase.tool.CanaryTool$RegionMonitor.run(CanaryTool.java:1366)
> at java.base/java.lang.Thread.run(Thread.java:834)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
Apache-HBase commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549125040 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 34s | branch-2.2 passed | | :green_heart: | compile | 1m 13s | branch-2.2 passed | | :green_heart: | checkstyle | 1m 46s | branch-2.2 passed | | :green_heart: | shadedjars | 4m 21s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 51s | branch-2.2 passed | | :blue_heart: | spotbugs | 3m 13s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 3m 47s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 11s | the patch passed | | :green_heart: | compile | 1m 14s | the patch passed | | :green_heart: | javac | 1m 14s | the patch passed | | :green_heart: | checkstyle | 1m 42s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 26s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 48s | the patch passed | | :green_heart: | findbugs | 4m 9s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 30s | hbase-procedure in the patch passed. | | :green_heart: | unit | 222m 19s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 50s | The patch does not generate ASF License warnings. | | | | 286m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/791 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 8898a66e46b7 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-791/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 180fa3eecc | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/2/testReport/ | | Max. process+thread count | 4372 (vs. ulimit of 1) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23055) Alter hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965634#comment-16965634
]
Hudson commented on HBASE-23055:
Results for branch HBASE-23055
[build #34 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23055/34/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23055/34//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23055/34//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-23055/34//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Alter hbase:meta
>
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
> Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0
>
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our
> being able to set encodings such as the block-with-indexes which will help
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first
> step on road to being able to split meta.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
Apache9 commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549114717 Maybe the final solution is to just drop the current ProcedureStore code and rewrite it completely... Too many unexpected exceptions... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache9 commented on a change in pull request #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
Apache9 commented on a change in pull request #791: HBASE-23247 [hbck2]
Schedule SCPs for 'Unknown Servers'
URL: https://github.com/apache/hbase/pull/791#discussion_r341840235
##
File path:
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/ProcedureStoreTracker.java
##
@@ -136,6 +136,9 @@ private BitSetNode update(BitSetNode node, long procId) {
node = lookupClosestNode(node, procId);
assert node != null : "expected node to update procId=" + procId;
Review comment:
We have an assert here on node not null?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #790: HBASE-23243 [pv2] Filter out SUCCESS procedures; on decent-sized clus…
Apache-HBase commented on issue #790: HBASE-23243 [pv2] Filter out SUCCESS procedures; on decent-sized clus… URL: https://github.com/apache/hbase/pull/790#issuecomment-549114080 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | :green_heart: | mvninstall | 5m 12s | branch-2.2 passed | | :green_heart: | javadoc | 0m 38s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 4m 43s | the patch passed | | :broken_heart: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | :green_heart: | javadoc | 0m 34s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 265m 17s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 279m 34s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-790/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/790 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 3956c6891149 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-790/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 180fa3eecc | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-790/2/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-790/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-790/2/testReport/ | | Max. process+thread count | 5054 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-790/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20395) Displaying thrift server type on the thrift page
[
https://issues.apache.org/jira/browse/HBASE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965606#comment-16965606
]
HBase QA commented on HBASE-20395:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red} HBASE-20395 does not apply to master. Rebase required? Wrong
Branch? See
https://yetus.apache.org/documentation/in-progress/precommit-patchnames for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20395 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12919522/HBASE-20395.master.005.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/986/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.11.0 https://yetus.apache.org |
This message was automatically generated.
> Displaying thrift server type on the thrift page
>
>
> Key: HBASE-20395
> URL: https://issues.apache.org/jira/browse/HBASE-20395
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
>Priority: Major
> Attachments: HBASE-20395.master.001.patch,
> HBASE-20395.master.002.patch, HBASE-20395.master.003.patch,
> HBASE-20395.master.004.patch, HBASE-20395.master.005.patch,
> HBASE-20395.master.005.patch, result.png
>
>
> HBase supports two types of thrift server: thrift and thrift2.
> But after start the thrift server successfully, we can not get the thrift
> server type conveniently.
> So, displaying thrift server type on the thrift page may provide some
> convenience for the users.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-20395) Displaying thrift server type on the thrift page
[ https://issues.apache.org/jira/browse/HBASE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965603#comment-16965603 ] Guangxu Cheng commented on HBASE-20395: --- the patch can't apply to master now. let me rebase and generate another patch for it. > Displaying thrift server type on the thrift page > > > Key: HBASE-20395 > URL: https://issues.apache.org/jira/browse/HBASE-20395 > Project: HBase > Issue Type: Improvement > Components: Thrift >Reporter: Guangxu Cheng >Assignee: Guangxu Cheng >Priority: Major > Attachments: HBASE-20395.master.001.patch, > HBASE-20395.master.002.patch, HBASE-20395.master.003.patch, > HBASE-20395.master.004.patch, HBASE-20395.master.005.patch, > HBASE-20395.master.005.patch, result.png > > > HBase supports two types of thrift server: thrift and thrift2. > But after start the thrift server successfully, we can not get the thrift > server type conveniently. > So, displaying thrift server type on the thrift page may provide some > convenience for the users. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r341837096 ## File path: hbase-shell/src/main/ruby/shell.rb ## @@ -333,10 +333,12 @@ def help_footer normalizer_switch normalizer_enabled is_in_maintenance_mode +clear_slow_responses Review comment: Is this fine with you @guangxuCheng ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog…
virajjasani commented on issue #787: HBASE-23245 : MutableHistogram constructor changes and provide Histog… URL: https://github.com/apache/hbase/pull/787#issuecomment-549109204 Please review @enis @elliottneilclark This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965593#comment-16965593
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #168 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/168/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/168//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/168//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/168//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers'
Apache-HBase commented on issue #791: HBASE-23247 [hbck2] Schedule SCPs for 'Unknown Servers' URL: https://github.com/apache/hbase/pull/791#issuecomment-549108412 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-2.2 Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 34s | branch-2.2 passed | | :green_heart: | compile | 1m 13s | branch-2.2 passed | | :green_heart: | checkstyle | 1m 45s | branch-2.2 passed | | :green_heart: | shadedjars | 4m 23s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 50s | branch-2.2 passed | | :blue_heart: | spotbugs | 3m 14s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 3m 46s | branch-2.2 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 12s | the patch passed | | :green_heart: | compile | 1m 15s | the patch passed | | :green_heart: | javac | 1m 15s | the patch passed | | :green_heart: | checkstyle | 1m 39s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 25s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 29s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 49s | the patch passed | | :green_heart: | findbugs | 3m 51s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 36s | hbase-procedure in the patch passed. | | :green_heart: | unit | 219m 27s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 283m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/791 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a13d68b1eae7 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-791/out/precommit/personality/provided.sh | | git revision | branch-2.2 / 180fa3eecc | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/1/testReport/ | | Max. process+thread count | 4514 (vs. ulimit of 1) | | modules | C: hbase-procedure hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-791/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
