[jira] [Commented] (HBASE-23567) TestReplicationStatus Suspicious Assertion
[
https://issues.apache.org/jira/browse/HBASE-23567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042803#comment-17042803
]
Michael Stack commented on HBASE-23567:
---
Let me fix this over in HBASE-23885.
> TestReplicationStatus Suspicious Assertion
> --
>
> Key: HBASE-23567
> URL: https://issues.apache.org/jira/browse/HBASE-23567
> Project: HBase
> Issue Type: Task
>Reporter: David Mollitor
>Priority: Minor
>
> {code:java|TestReplicationStatus.java}
> ServerMetrics sm = metrics.getLiveServerMetrics().get(server);
> List rLoadSourceList =
> sm.getReplicationLoadSourceList();
> // check SourceList still only has one entry
> assertTrue("failed to get ReplicationLoadSourceList",
> (rLoadSourceList.size() == 2));
> assertEquals(PEER_ID2, rLoadSourceList.get(0).getPeerID());
> {code}
> Claims that the sourceList should only have one entry and asserts that the
> one entry is {{PEER_ID2}} but it also asserts that the list should have two
> items in it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-23885) [Flakey Test] TestReplicationStatus
Michael Stack created HBASE-23885: - Summary: [Flakey Test] TestReplicationStatus Key: HBASE-23885 URL: https://issues.apache.org/jira/browse/HBASE-23885 Project: HBase Issue Type: Bug Components: flakies Reporter: Michael Stack Fix For: 3.0.0, 2.3.0 Fails about 20% of the time even when i run locally. Spent time trying to untangle but this is an awkward one. It is old. Subclasses TestReplicationBase which itself needs cleanup. Tried adding barriers to wait on events but holding up main thread messes up the test. Needs deeper dive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23886) FileSystemUtilizationChore reduces unnecessary regions
Bo Cui created HBASE-23886: -- Summary: FileSystemUtilizationChore reduces unnecessary regions Key: HBASE-23886 URL: https://issues.apache.org/jira/browse/HBASE-23886 Project: HBase Issue Type: Improvement Components: Quotas Affects Versions: 2.2.3 Reporter: Bo Cui Attachments: image-2020-02-23-13-59-32-894.png If the RS has 2000 reegions, and only one region1 has been disk space quota. FileSystemUtilizationChore can reduce unnecessary regions. RS only need to collect the region1 size !image-2020-02-23-13-59-32-894.png|width=608,height=138! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042801#comment-17042801
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #282 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/282/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/282//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/282//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/282//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23120) Fix ref guide - Release process
[
https://issues.apache.org/jira/browse/HBASE-23120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042794#comment-17042794
]
Sakthi commented on HBASE-23120:
'+$ git tag -s rel/2.0.0-alpha4 -m "Tagging the 2.0.0-alpha4 Release"' has a +
sign at the start. Need to fix it.
> Fix ref guide - Release process
> ---
>
> Key: HBASE-23120
> URL: https://issues.apache.org/jira/browse/HBASE-23120
> Project: HBase
> Issue Type: Bug
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Minor
> Fix For: 3.0.0
>
>
> The example is of a sample ~/.m2/settings.xml file.
> {code:java}
>
> {code}
> is not a proper comment.
> Also I think here,
> {code:java}
>
> {code}
> the double hyphened --list-keys shouldn't be used. It leads to FATAL error
> like the one below:
> {code:java}
> [FATAL] Non-parseable settings
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1197: [WIP-don't-merge] HBASE-18095: Zookeeper-less client connection umbrella [branch-2]
Apache-HBase commented on issue #1197: [WIP-don't-merge] HBASE-18095: Zookeeper-less client connection umbrella [branch-2] URL: https://github.com/apache/hbase/pull/1197#issuecomment-590024120 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 3s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 90 new or modified test files. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 18s | branch-2 passed | | +1 :green_heart: | compile | 3m 18s | branch-2 passed | | +1 :green_heart: | checkstyle | 3m 14s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 16s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 11s | branch-2 passed | | +0 :ok: | spotbugs | 0m 35s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 9m 15s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 55s | the patch passed | | +1 :green_heart: | compile | 3m 14s | the patch passed | | +1 :green_heart: | cc | 3m 14s | the patch passed | | +1 :green_heart: | javac | 3m 14s | the patch passed | | +1 :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 0m 25s | hbase-common: The patch generated 0 new + 16 unchanged - 2 fixed = 16 total (was 18) | | +1 :green_heart: | checkstyle | 0m 40s | hbase-client: The patch generated 0 new + 262 unchanged - 24 fixed = 262 total (was 286) | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-zookeeper | | -1 :x: | checkstyle | 1m 26s | hbase-server: The patch generated 2 new + 609 unchanged - 67 fixed = 611 total (was 676) | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-endpoint | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 17s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 38s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 3m 10s | the patch passed | | +1 :green_heart: | javadoc | 2m 9s | the patch passed | | +1 :green_heart: | findbugs | 10m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 42s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 9s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 2m 10s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 42s | hbase-zookeeper in the patch passed. | | -1 :x: | unit | 67m 51s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 2m 31s | hbase-endpoint in the patch passed. | | +1 :green_heart: | asflicense | 3m 13s | The patch does not generate ASF License warnings. | | | | 162m 5s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.TestZooKeeper | | | hadoop.hbase.client.TestMobRestoreSnapshotFromClientAfterSplittingRegions | | | hadoop.hbase.client.TestZKConnectionRegistry | | | hadoop.hbase.replication.regionserver.TestGlobalReplicationThrottler | | | hadoop.hbase.security.visibility.TestVisibilityLabelsReplication | | | hadoop.hbase.client.replication.TestReplicationAdminWithClusters | | | hadoop.hbase.client.TestAdmin2 | | | hadoop.hbase.replication.TestReplicationWithTags | | | hadoop.hbase.security.visibility.TestVisibilityLabelReplicationWithExpAsString | | | hadoop.hbase.security.provider.TestCustomSaslAuthenticationProvider | | | hadoop.hbase.client.TestReplicasClient | | | hadoop.hbase.client.TestScannersFromClientSide | | | hadoop.hbase.replication.multiwal.TestReplicationEndpointWithMultipleAsyncWAL | | | hadoop.hbase.replication.regionserver.TestReplicator | | | hadoop.hbase.client.TestConnectionImplementation | | Subsystem |
[jira] [Comment Edited] (HBASE-23743) Release 1.4.13
[ https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042773#comment-17042773 ] Sakthi edited comment on HBASE-23743 at 2/23/20 1:59 AM: - Update poms, changes.txt. Pushed the RC tag 1.4.13RC0 was (Author: jatsakthi): Update poms, changes.txt. Pushed the release tag 1.4.13RC0 > Release 1.4.13 > -- > > Key: HBASE-23743 > URL: https://issues.apache.org/jira/browse/HBASE-23743 > Project: HBase > Issue Type: Task > Components: build, community >Affects Versions: 1.4.13 >Reporter: Sean Busbey >Assignee: Sakthi >Priority: Major > Fix For: 1.4.13 > > > over due for 1.4.13. I'll try to use the release manager scripts this time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23743) Release 1.4.13
[ https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042773#comment-17042773 ] Sakthi commented on HBASE-23743: Update poms, changes.txt. Pushed the release tag 1.4.13RC0 > Release 1.4.13 > -- > > Key: HBASE-23743 > URL: https://issues.apache.org/jira/browse/HBASE-23743 > Project: HBase > Issue Type: Task > Components: build, community >Affects Versions: 1.4.13 >Reporter: Sean Busbey >Assignee: Sakthi >Priority: Major > Fix For: 1.4.13 > > > over due for 1.4.13. I'll try to use the release manager scripts this time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23884) list_rsgroups didn't get the correct result
[ https://issues.apache.org/jira/browse/HBASE-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HBASE-23884: --- Priority: Minor (was: Major) > list_rsgroups didn't get the correct result > --- > > Key: HBASE-23884 > URL: https://issues.apache.org/jira/browse/HBASE-23884 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 2.2.3 >Reporter: Bo Cui >Priority: Minor > Attachments: image-2020-02-23-09-53-49-594.png > > > if my_group does not exist, list_rsgroups will get 0 row(s), > but i think list_rsgroups should throw NOTEXISTEXCEPTION > !image-2020-02-23-09-53-49-594.png|width=449,height=143! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23884) list_rsgroups didn't get the correct result
Bo Cui created HBASE-23884: -- Summary: list_rsgroups didn't get the correct result Key: HBASE-23884 URL: https://issues.apache.org/jira/browse/HBASE-23884 Project: HBase Issue Type: Bug Components: rsgroup, shell Affects Versions: 2.2.3 Reporter: Bo Cui Attachments: image-2020-02-23-09-53-49-594.png in hbase shell, if my_group does not exist, list_rsgroups will get 0 row(s), but i think list_rsgroups should throw NOTEXISTEXCEPTION !image-2020-02-23-09-53-49-594.png|width=449,height=143! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23884) list_rsgroups didn't get the correct result
[ https://issues.apache.org/jira/browse/HBASE-23884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Cui updated HBASE-23884: --- Description: if my_group does not exist, list_rsgroups will get 0 row(s), but i think list_rsgroups should throw NOTEXISTEXCEPTION !image-2020-02-23-09-53-49-594.png|width=449,height=143! was: in hbase shell, if my_group does not exist, list_rsgroups will get 0 row(s), but i think list_rsgroups should throw NOTEXISTEXCEPTION !image-2020-02-23-09-53-49-594.png|width=449,height=143! > list_rsgroups didn't get the correct result > --- > > Key: HBASE-23884 > URL: https://issues.apache.org/jira/browse/HBASE-23884 > Project: HBase > Issue Type: Bug > Components: rsgroup, shell >Affects Versions: 2.2.3 >Reporter: Bo Cui >Priority: Major > Attachments: image-2020-02-23-09-53-49-594.png > > > if my_group does not exist, list_rsgroups will get 0 row(s), > but i think list_rsgroups should throw NOTEXISTEXCEPTION > !image-2020-02-23-09-53-49-594.png|width=449,height=143! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
Apache-HBase commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590016976 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 1m 2s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 8m 1s | master passed | | +1 :green_heart: | compile | 4m 48s | master passed | | +1 :green_heart: | checkstyle | 3m 16s | master passed | | +0 :ok: | refguide | 7m 36s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 5m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 3m 50s | master passed | | +0 :ok: | spotbugs | 15m 57s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 17m 40s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 47s | the patch passed | | +1 :green_heart: | compile | 3m 42s | the patch passed | | +1 :green_heart: | javac | 3m 42s | the patch passed | | +1 :green_heart: | checkstyle | 2m 19s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 4m 46s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 9s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 3m 9s | the patch passed | | +1 :green_heart: | findbugs | 16m 28s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 62m 24s | root in the patch failed. | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 181m 20s | | | Reason | Tests | |---:|:--| | Unreaped Processes | root:8 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1186 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml refguide | | uname | Linux 148725ee8410 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1186/out/precommit/personality/provided.sh | | git revision | master / 400b7ce954 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/artifact/out/patch-site/book.html | | Unreaped Processes Log | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/artifact/out/patch-unit-root-reaper.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/testReport/ | | Max. process+thread count | 9698 (vs. ulimit of 1) | | modules | C: hbase-protocol . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1186/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated.
[jira] [Commented] (HBASE-23743) Release 1.4.13
[ https://issues.apache.org/jira/browse/HBASE-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042772#comment-17042772 ] Sakthi commented on HBASE-23743: Gira and git status look intact. > Release 1.4.13 > -- > > Key: HBASE-23743 > URL: https://issues.apache.org/jira/browse/HBASE-23743 > Project: HBase > Issue Type: Task > Components: build, community >Affects Versions: 1.4.13 >Reporter: Sean Busbey >Assignee: Sakthi >Priority: Major > Fix For: 1.4.13 > > > over due for 1.4.13. I'll try to use the release manager scripts this time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] bharathv opened a new pull request #1197: [WIP-don't-merge] HBASE-18095: Zookeeper-less client connection umbrella [branch-2]
bharathv opened a new pull request #1197: [WIP-don't-merge] HBASE-18095: Zookeeper-less client connection umbrella [branch-2] URL: https://github.com/apache/hbase/pull/1197 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1196: HBASE-23755. [OpenTracing] Declare HTrace is unusable in the user doc
Apache-HBase commented on issue #1196: HBASE-23755. [OpenTracing] Declare HTrace is unusable in the user doc URL: https://github.com/apache/hbase/pull/1196#issuecomment-590014698 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 38s | master passed | | +0 :ok: | refguide | 4m 52s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 4m 45s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 20s | The patch does not generate ASF License warnings. | | | | 22m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1196/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1196 | | Optional Tests | dupname asflicense refguide | | uname | Linux 6fcfce720ef5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1196/out/precommit/personality/provided.sh | | git revision | master / 400b7ce954 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1196/2/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1196/2/artifact/out/patch-site/book.html | | Max. process+thread count | 96 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1196/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23862) Fix flaky TestSnapshotFromMaster in 1.x versions
[ https://issues.apache.org/jira/browse/HBASE-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-23862: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix flaky TestSnapshotFromMaster in 1.x versions > > > Key: HBASE-23862 > URL: https://issues.apache.org/jira/browse/HBASE-23862 > Project: HBase > Issue Type: Task >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Fix For: 1.3.7, 1.7.0, 1.4.13 > > Attachments: hbase-23862.branch-1.3.001.patch, > hbase-23862.branch-1.4.001.patch > > > HBASE-23658 was about fixing this flakey ut in branch-2+. I think it can be > backported to branch-1s as well. 18/30 times this test has failed in > branch-1, 10/30 in 1.4 and 3/30 in 1.3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23854) Documentation update of external_apis.adoc#example-scala-code
[
https://issues.apache.org/jira/browse/HBASE-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042758#comment-17042758
]
Hudson commented on HBASE-23854:
Results for branch branch-2.2
[build #795 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/795/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/795//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/795//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/795//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Documentation update of external_apis.adoc#example-scala-code
> -
>
> Key: HBASE-23854
> URL: https://issues.apache.org/jira/browse/HBASE-23854
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Michael Heil
>Assignee: Michael Heil
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: HBASE-23854.patch
>
>
> Update the Example Scala Code in the Reference Guide as it contains
> deprecated content such as
> * new HBaseConfiguration()
> * new HTable(conf, "mytable")
> * add(Bytes.toBytes("ids"),Bytes.toBytes("id1"),Bytes.toBytes("one"))
> Replace it with:
> * HBaseConfiguration.create()
> * TableName.valueOf({color:#6a8759}"mytable"{color})
> *
> addColumn(Bytes.toBytes({color:#6a8759}"ids"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"id1"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"one"{color}))
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590005567 CP == Coprocessor CPEP == CoprocessorEndpoint TMI == Too Much Information This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590005048 > Purging mention of HBase ZeroCopy from doc would be good, yes. Done! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
[
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042733#comment-17042733
]
Martin Tzvetanov Grigorov commented on HBASE-19146:
---
Thanks! I will know for the next PR! :-)
> Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
> Issue Type: Bug
> Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS: Ubuntu 16.04.3
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform: AARCH64
>Reporter: Yuqi Gu
>Priority: Major
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
> com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
>
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
>
>false
>true
>
>
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> -
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
> + /usr/local/bin/protoc
> {code}
> The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
> identifier "proto3". This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590004718 The mail discussion is in favour of upgrading to PB3! What do you mean with `to do TMI` ? I am not aware of this abbreviation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] martin-g commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#discussion_r382945144 ## File path: pom.xml ## @@ -1517,7 +1517,7 @@ 1.2.17 2.28.2 -2.5.0 +3.11.4 Review comment: The idea is to upgrade PB2.5 to PB3.x in HBase 3.x. Now is the best moment to do this upgrade. As discussed at https://issues.apache.org/jira/browse/HBASE-16761 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1189: HBASE-23842 Removed deprecated Scan(byte[], byte[]) from Scan
Apache-HBase commented on issue #1189: HBASE-23842 Removed deprecated Scan(byte[], byte[]) from Scan URL: https://github.com/apache/hbase/pull/1189#issuecomment-590004331 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 18 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 17s | master passed | | +1 :green_heart: | compile | 3m 52s | master passed | | +1 :green_heart: | checkstyle | 3m 35s | master passed | | +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 48s | master passed | | +0 :ok: | spotbugs | 1m 32s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 9m 5s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 3m 49s | the patch passed | | +1 :green_heart: | javac | 3m 49s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | The patch passed checkstyle in hbase-client | | +1 :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 374 unchanged - 83 fixed = 374 total (was 457) | | +1 :green_heart: | checkstyle | 0m 20s | The patch passed checkstyle in hbase-mapreduce | | +1 :green_heart: | checkstyle | 0m 38s | The patch passed checkstyle in hbase-thrift | | +1 :green_heart: | checkstyle | 0m 13s | The patch passed checkstyle in hbase-shell | | +1 :green_heart: | checkstyle | 0m 15s | The patch passed checkstyle in hbase-endpoint | | +1 :green_heart: | checkstyle | 0m 17s | The patch passed checkstyle in hbase-rest | | +1 :green_heart: | rubocop | 0m 3s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 55s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 2m 44s | the patch passed | | +1 :green_heart: | findbugs | 10m 8s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 55s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 62m 41s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 8m 36s | hbase-mapreduce in the patch passed. | | +1 :green_heart: | unit | 2m 30s | hbase-thrift in the patch passed. | | +1 :green_heart: | unit | 7m 19s | hbase-shell in the patch passed. | | +1 :green_heart: | unit | 3m 9s | hbase-endpoint in the patch passed. | | +1 :green_heart: | unit | 3m 7s | hbase-rest in the patch passed. | | +1 :green_heart: | asflicense | 3m 38s | The patch does not generate ASF License warnings. | | | | 176m 11s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1189/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1189 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile rubocop | | uname | Linux 1ff6dcde3b3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1189/out/precommit/personality/provided.sh | | git revision | master / 400b7ce954 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1189/4/testReport/ | | Max. process+thread count | 8634 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server hbase-mapreduce hbase-thrift hbase-shell hbase-endpoint hbase-rest U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1189/4/console | |
[GitHub] [hbase] martin-g removed a comment on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g removed a comment on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590003017 What are CP and CPEP in the mail discussion ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-590003017 What are CP and CPEP in the mail discussion ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] martin-g commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4
martin-g commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#discussion_r382945144 ## File path: pom.xml ## @@ -1517,7 +1517,7 @@ 1.2.17 2.28.2 -2.5.0 +3.11.4 Review comment: The idea is to upgrade PB2.5 to PB3.x in HBase 3.x. Now is the best moment to do this upgrade. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23862) Fix flaky TestSnapshotFromMaster in 1.x versions
[
https://issues.apache.org/jira/browse/HBASE-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042731#comment-17042731
]
Hudson commented on HBASE-23862:
Results for branch branch-1.3
[build #1140 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1140/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1140//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1140//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.3/1140//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix flaky TestSnapshotFromMaster in 1.x versions
>
>
> Key: HBASE-23862
> URL: https://issues.apache.org/jira/browse/HBASE-23862
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.3.7, 1.7.0, 1.4.13
>
> Attachments: hbase-23862.branch-1.3.001.patch,
> hbase-23862.branch-1.4.001.patch
>
>
> HBASE-23658 was about fixing this flakey ut in branch-2+. I think it can be
> backported to branch-1s as well. 18/30 times this test has failed in
> branch-1, 10/30 in 1.4 and 3/30 in 1.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[
https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042704#comment-17042704
]
Hudson commented on HBASE-23864:
Results for branch branch-2
[build #2499 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2499/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2499//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2499//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2499//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when
> split/merge is disabled
> ---
>
> Key: HBASE-23864
> URL: https://issues.apache.org/jira/browse/HBASE-23864
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG
>
>
> Now even the split/merge is disabled, master will submit a
> SplitTableRegionProcedure, too. And rollback it when execute failed. I
> thought the split/merge switch is a cluster level swtich. Master can check it
> early and no need to submit
> SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch
> is disabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23862) Fix flaky TestSnapshotFromMaster in 1.x versions
[
https://issues.apache.org/jira/browse/HBASE-23862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042697#comment-17042697
]
Hudson commented on HBASE-23862:
Results for branch branch-1.4
[build #1187 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1187/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1187//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1187//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1.4/1187//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix flaky TestSnapshotFromMaster in 1.x versions
>
>
> Key: HBASE-23862
> URL: https://issues.apache.org/jira/browse/HBASE-23862
> Project: HBase
> Issue Type: Task
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.3.7, 1.7.0, 1.4.13
>
> Attachments: hbase-23862.branch-1.3.001.patch,
> hbase-23862.branch-1.4.001.patch
>
>
> HBASE-23658 was about fixing this flakey ut in branch-2+. I think it can be
> backported to branch-1s as well. 18/30 times this test has failed in
> branch-1, 10/30 in 1.4 and 3/30 in 1.3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-589977053 @martin-g The 'other work' I refer to above that is moving to update our pb2.5 is being discussed on this dev list thread https://www.mail-archive.com/dev@hbase.apache.org/msg56186.html I am not trying to do TMI. Just citing background in case you are interested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23854) Documentation update of external_apis.adoc#example-scala-code
[
https://issues.apache.org/jira/browse/HBASE-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042645#comment-17042645
]
Michael Heil commented on HBASE-23854:
--
Hi [~busbey] , thank you for your comment and letting me know how to best
approach the reviewers!
> Documentation update of external_apis.adoc#example-scala-code
> -
>
> Key: HBASE-23854
> URL: https://issues.apache.org/jira/browse/HBASE-23854
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Michael Heil
>Assignee: Michael Heil
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: HBASE-23854.patch
>
>
> Update the Example Scala Code in the Reference Guide as it contains
> deprecated content such as
> * new HBaseConfiguration()
> * new HTable(conf, "mytable")
> * add(Bytes.toBytes("ids"),Bytes.toBytes("id1"),Bytes.toBytes("one"))
> Replace it with:
> * HBaseConfiguration.create()
> * TableName.valueOf({color:#6a8759}"mytable"{color})
> *
> addColumn(Bytes.toBytes({color:#6a8759}"ids"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"id1"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"one"{color}))
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-thirdparty] busbey commented on a change in pull request #13: HBASE-23811. [OpenTracing] Add shaded JaegerTracing tracer to hbase-thirdparty.
busbey commented on a change in pull request #13: HBASE-23811. [OpenTracing]
Add shaded JaegerTracing tracer to hbase-thirdparty.
URL: https://github.com/apache/hbase-thirdparty/pull/13#discussion_r382926913
##
File path: hbase-shaded-jaeger/pom.xml
##
@@ -0,0 +1,176 @@
+
+https://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance;
+ xsi:schemaLocation="https://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd;>
+
+4.0.0
+
+hbase-thirdparty
+org.apache.hbase.thirdparty
+3.3.0-SNAPSHOT
+..
+
+hbase-shaded-jaeger
+Apache HBase Relocated (Shaded) JaegerTracing Libs
+The shaded uber jar of JaegerTracing library used by
HBase.
+
+
+
+io.jaegertracing
+jaeger-client
+${jaegertracing.version}
+
+
+
+
+
+
+
+
+maven-assembly-plugin
+
+true
+
+
+
+org.apache.maven.plugins
+maven-shade-plugin
+
+
+
aggregate-into-a-jar-with-relocated-third-parties
+package
+
+shade
+
+
+false
+
false
+false
+
+
+
+
org.apache.httpcomponents:httpclient
+
commons-logging:commons-logging
+
commons-codec:commons-codec
+
org.apache.httpcomponents:httpcore
+
javax.annotation:javax.annotation-api
+org.slf4j:slf4j-api
+io.opentracing:*
+
+
+
+
+
Review comment:
Probably not worth trying to deduplicate with the shaded gson we use?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4
saintstack commented on issue #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#issuecomment-589976428 Purging mention of HBase ZeroCopy from doc would be good, yes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4
saintstack commented on a change in pull request #1186: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase/pull/1186#discussion_r382926536 ## File path: pom.xml ## @@ -1517,7 +1517,7 @@ 1.2.17 2.28.2 -2.5.0 +3.11.4 Review comment: Ouch. Do you get the bit where we use shaded pb3 internally but that unshaded pb2.5 is available for users? (HBase3/master branch is looking like it will purge the unshaded pb2.5 usage... but that is another PR). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22978) Online slow response log
[
https://issues.apache.org/jira/browse/HBASE-22978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042638#comment-17042638
]
Viraj Jasani commented on HBASE-22978:
--
Update on the progress:
I have addressed all concerns in latest reviews, almost ready for commit to
master branch and then will prepare branch-2 backport. Will create all follow
up tasks soon.
> Online slow response log
>
>
> Key: HBASE-22978
> URL: https://issues.apache.org/jira/browse/HBASE-22978
> Project: HBase
> Issue Type: New Feature
> Components: Admin, Operability, regionserver, shell
>Affects Versions: 3.0.0, 2.3.0, 1.5.1
>Reporter: Andrew Kyle Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 1.7.0
>
> Attachments: Screen Shot 2019-10-19 at 2.31.59 AM.png, Screen Shot
> 2019-10-19 at 2.32.54 AM.png, Screen Shot 2019-10-19 at 2.34.11 AM.png,
> Screen Shot 2019-10-19 at 2.36.14 AM.png
>
>
> Today when an individual RPC exceeds a configurable time bound we log a
> complaint by way of the logging subsystem. These log lines look like:
> {noformat}
> 2019-08-30 22:10:36,195 WARN [,queue=15,port=60020] ipc.RpcServer -
> (responseTooSlow):
> {"call":"Scan(org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ScanRequest)",
> "starttimems":1567203007549,
> "responsesize":6819737,
> "method":"Scan",
> "param":"region { type: REGION_NAME value:
> \"tsdb,\\000\\000\\215\\f)o\\024\\302\\220\\000\\000\\000\\000\\000\\001\\000\\000\\000\\000\\000\\006\\000\\000\\000\\000\\000\\005\\000\\000",
> "processingtimems":28646,
> "client":"10.253.196.215:41116",
> "queuetimems":22453,
> "class":"HRegionServer"}
> {noformat}
> Unfortunately we often truncate the request parameters, like in the above
> example. We do this because the human readable representation is verbose, the
> rate of too slow warnings may be high, and the combination of these things
> can overwhelm the log capture system. The truncation is unfortunate because
> it eliminates much of the utility of the warnings. For example, the region
> name, the start and end keys, and the filter hierarchy are all important
> clues for debugging performance problems caused by moderate to low
> selectivity queries or queries made at a high rate.
> We can maintain an in-memory ring buffer of requests that were judged to be
> too slow in addition to the responseTooSlow logging. The in-memory
> representation can be complete and compressed. A new admin API and shell
> command can provide access to the ring buffer for online performance
> debugging. A modest sizing of the ring buffer will prevent excessive memory
> utilization for a minor performance debugging feature by limiting the total
> number of retained records. There is some chance a high rate of requests will
> cause information on other interesting requests to be overwritten before it
> can be read. This is the nature of a ring buffer and an acceptable trade off.
> The write request types do not require us to retain all information submitted
> in the request. We don't need to retain all key-values in the mutation, which
> may be too large to comfortably retain. We only need a unique set of row
> keys, or even a min/max range, and total counts.
> The consumers of this information will be debugging tools. We can afford to
> apply fast compression to ring buffer entries (if codec support is
> available), something like snappy or zstandard, and decompress on the fly
> when servicing the retrieval API request. This will minimize the impact of
> retaining more information about slow requests than we do today.
> This proposal is for retention of request information only, the same
> information provided by responseTooSlow warnings. Total size of response
> serialization, possibly also total cell or row counts, should be sufficient
> to characterize the response.
> Optionally persist new entries added to the ring buffer into one or more
> files in HDFS in a write-behind manner. If the HDFS writer blocks or falls
> behind and we are unable to persist an entry before it is overwritten, that
> is fine. Response too slow logging is best effort. If we can detect this make
> a note of it in the log file. Provide a tool for parsing, dumping, filtering,
> and pretty printing the slow logs written to HDFS. The tool and the shell can
> share and reuse some utility classes and methods for accomplishing that.
> —
> New shell commands:
> {{get_slow_responses [ ... , ] [ , \{
> } ]}}
> Retrieve, decode, and pretty print the contents of the too slow response ring
> buffer maintained by the given list of servers; or all servers in the cluster
> if no list is provided. Optionally provide a map of parameters for filtering
> as additional argument. The TABLE filter, which expects
[jira] [Commented] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
[
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042633#comment-17042633
]
Michael Stack commented on HBASE-19146:
---
[~mgrigorov] I made a mistake by merging your PR. My fault. Our practice is one
JIRA per PR. I should have let you know this and asked you to open a new JIRA
(perhaps a subissue of this one) to host your pb update PR. For next time.
Meantime, thanks for the patch.
> Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
> Issue Type: Bug
> Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS: Ubuntu 16.04.3
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform: AARCH64
>Reporter: Yuqi Gu
>Priority: Major
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
> com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
>
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
>
>false
>true
>
>
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> -
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
> + /usr/local/bin/protoc
> {code}
> The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
> identifier "proto3". This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] busbey commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase
busbey commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase URL: https://github.com/apache/hbase/pull/1165#issuecomment-589975448 Unit test failures look related? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-19146) Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
[
https://issues.apache.org/jira/browse/HBASE-19146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042630#comment-17042630
]
Michael Stack commented on HBASE-19146:
---
Merged https://github.com/apache/hbase-thirdparty/pull/14 which upgrades the pb
version used and fixes pom complaint.
> Hbase3.0 protobuf-maven-plugin do not support Arm64(only for x86)
> --
>
> Key: HBASE-19146
> URL: https://issues.apache.org/jira/browse/HBASE-19146
> Project: HBase
> Issue Type: Bug
> Components: build, pom
>Affects Versions: 3.0.0
> Environment: OS: Ubuntu 16.04.3
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> Hw platform: AARCH64
>Reporter: Yuqi Gu
>Priority: Major
>
> We are building the HBase 3.0.0-SNAPSHOT on AARCH64.
> It is noted that 'protobuf-maven-plugin' only support x86 shown as follows:
> {code:java}
>
>org.xolstice.maven.plugins
>protobuf-maven-plugin
>${protobuf.plugin.version}
>
> com.google.protobuf:protoc:${external.protobuf.version}:
> exe:${os.detected.classifier}
>
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
>
>false
>true
>
>
> {code}
> So the build is failed.
> {code:java}
> [INFO] --- protobuf-maven-plugin:0.5.0:compile (compile-protoc) @
> hbase-protocol-shaded ---
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> Failed to execute goal
> org.xolstice.maven.plugins:protobuf-maven-plugin:0.5.0:compile
> (compile-protoc) on project hbase-protocol-shaded: Missing:
> {code}
> Then I installed aarch64 protobuf 2.5.0 on the host and modify the pom:
> {code:java}
> -
> com.google.protobuf:protoc:${external.protobuf.version}:exe:${os.detected.classifier}
> + /usr/local/bin/protoc
> {code}
> The build is also failed:
> {code:java}
> [INFO] Compiling 32 proto file(s) to
> /root/hbase/hbase-protocol-shaded/target/generated-sources/protobuf/java
> [ERROR] PROTOC FAILED: google/protobuf/any.proto:31:10: Unrecognized syntax
> identifier "proto3". This parser only recognizes "proto2".
> {code}
> It seems that "internal.protobuf.version" in "hbase-protocol-shaded" is 3.3.0.
> How to fix it? Thanks!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase-thirdparty] saintstack merged pull request #14: HBASE-19146 Update Protobuf to 3.11.4
saintstack merged pull request #14: HBASE-19146 Update Protobuf to 3.11.4 URL: https://github.com/apache/hbase-thirdparty/pull/14 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382925202
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#discussion_r382924890 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java ## @@ -3874,4 +3878,62 @@ private void getProcedureResult(long procId, CompletableFuture future, int .call(); } + @Override + public CompletableFuture> getSlowLogResponses( +@Nullable final Set serverNames, Review comment: Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382895260
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] HorizonNet commented on issue #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility
HorizonNet commented on issue #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility URL: https://github.com/apache/hbase/pull/1146#issuecomment-589972808 @ndimiduk @virajjasani Does your +1 stay? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet commented on issue #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility
HorizonNet commented on issue #1146: HBASE-23781 Removed deprecated createTableDescriptor(String) from HBaseTestingUtility URL: https://github.com/apache/hbase/pull/1146#issuecomment-589972737 Both test failures seem to be unrelated to the change. Ran them several times locally. `TestNamespaceReplicationWithBulkLoadedData` always passes, `TestReplicationStatus` 50% of the time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase
Apache-HBase commented on issue #1165: HBASE-22514 Move rsgroup feature into core of HBase URL: https://github.com/apache/hbase/pull/1165#issuecomment-589970621 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 2s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 47 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 47s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 14s | master passed | | +1 :green_heart: | compile | 3m 4s | master passed | | +1 :green_heart: | checkstyle | 2m 34s | master passed | | +1 :green_heart: | shadedjars | 4m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 6m 20s | master passed | | +0 :ok: | spotbugs | 1m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 16s | branch/hbase-annotations no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 32s | branch/hbase-assembly no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 2m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 3m 4s | the patch passed | | +1 :green_heart: | cc | 3m 4s | the patch passed | | +1 :green_heart: | javac | 3m 4s | the patch passed | | +1 :green_heart: | checkstyle | 2m 27s | root: The patch generated 0 new + 733 unchanged - 22 fixed = 733 total (was 755) | | +1 :green_heart: | rubocop | 0m 9s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 8s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 42s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 50s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 10m 25s | the patch passed | | +1 :green_heart: | javadoc | 6m 3s | the patch passed | | +0 :ok: | findbugs | 0m 13s | hbase-annotations has no data from findbugs | | +0 :ok: | findbugs | 0m 30s | hbase-assembly has no data from findbugs | ||| _ Other Tests _ | | -1 :x: | unit | 69m 35s | root in the patch failed. | | +1 :green_heart: | asflicense | 5m 38s | The patch does not generate ASF License warnings. | | | | 210m 7s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.rsgroup.TestRSGroupsBasics | | | hadoop.hbase.rsgroup.TestMigrateRSGroupInfo | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/1165 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml cc hbaseprotoc prototool rubocop | | uname | Linux 3f5aa5697a69 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/Base-PreCommit-GitHub-PR_PR-1165/out/precommit/personality/provided.sh | | git revision | master / 400b7ce954 | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/3/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/3/testReport/ | | Max. process+thread count | 8543 (vs. ulimit of 1) | | modules | C: hbase-annotations hbase-protocol-shaded hbase-common hbase-protocol hbase-client hbase-server hbase-thrift hbase-shell hbase-it hbase-assembly . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1165/3/console | | versions | git=2.11.0
[GitHub] [hbase] Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log
Apache-HBase commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-589966839 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 6 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 38s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 16s | master passed | | +1 :green_heart: | compile | 3m 5s | master passed | | +1 :green_heart: | checkstyle | 2m 26s | master passed | | +0 :ok: | refguide | 4m 55s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 5m 11s | master passed | | +0 :ok: | spotbugs | 1m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 23m 25s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 3m 6s | the patch passed | | +1 :green_heart: | cc | 3m 6s | the patch passed | | +1 :green_heart: | javac | 3m 6s | the patch passed | | +1 :green_heart: | checkstyle | 2m 24s | root: The patch generated 0 new + 433 unchanged - 1 fixed = 433 total (was 434) | | +1 :green_heart: | rubocop | 0m 10s | There were no new rubocop issues. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +0 :ok: | refguide | 4m 47s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 55s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | hbaseprotoc | 8m 25s | the patch passed | | +1 :green_heart: | javadoc | 5m 17s | the patch passed | | +1 :green_heart: | findbugs | 24m 53s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 73m 5s | root in the patch failed. | | +1 :green_heart: | asflicense | 3m 50s | The patch does not generate ASF License warnings. | | | | 209m 59s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestNamespaceReplicationWithBulkLoadedData | | | hadoop.hbase.replication.TestReplicationStatusAfterLagging | | | hadoop.hbase.replication.TestReplicationDisableInactivePeer | | | hadoop.hbase.replication.TestRemoveFromSerialReplicationPeer | | | hadoop.hbase.replication.TestSyncReplicationMoreLogsInLocalCopyToRemote | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.6 Server=19.03.6 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/20/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/754 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml cc hbaseprotoc prototool rubocop | | uname | Linux d7f88826477c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-754/out/precommit/personality/provided.sh | | git revision | master / 400b7ce954 | | Default Java | 1.8.0_181 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/20/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/20/artifact/out/patch-site/book.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-754/20/artifact/out/patch-unit-root.txt | | Test Results |
[GitHub] [hbase] busbey commented on a change in pull request #1165: HBASE-22514 Move rsgroup feature into core of HBase
busbey commented on a change in pull request #1165: HBASE-22514 Move rsgroup
feature into core of HBase
URL: https://github.com/apache/hbase/pull/1165#discussion_r382918299
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfo.java
##
@@ -87,7 +102,7 @@ public boolean containsServer(Address hostPort) {
/**
* Get list of servers.
*/
- public Set getServers() {
+ public SortedSet getServers() {
Review comment:
Can we avoid changing the method signature on this IA.Public method?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[
https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042583#comment-17042583
]
HBase QA commented on HBASE-23864:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
20s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 5m
10s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 58s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 97m
51s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.6 Server=19.03.6 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1139/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23864 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12994187/HBASE-23864-addendum.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0c82dd5dc6ad 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19
08:06:28 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 400b7ce954 |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/1139/testReport/
[jira] [Updated] (HBASE-22514) Move rsgroup feature into core of HBase
[ https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22514: -- Release Note: Moved rs group feature into core. Use this flag to enable or disable it. The coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint is deprected, but for compatible, if you want the pre 3.0.0 hbase client/shell to communicate with the new hbase cluster, you still need to add this coprocessor to master. And if this coprocessor is specified, the above flag will be set to true automatically to enable rs group feature. These methods are added to the Admin/AsyncAdmin interface for managing rs groups. See the javadoc of these methods for more details. void addRSGroup(String groupName) throws IOException; RSGroupInfo getRSGroup(String groupName) throws IOException; RSGroupInfo getRSGroup(Address hostPort) throws IOException; RSGroupInfo getRSGroup(TableName tableName) throws IOException; List listRSGroups() throws IOException; List listTablesInRSGroup(String groupName) throws IOException; Pair, List> getConfiguredNamespacesAndTablesInRSGroup(String groupName) throws IOException; void removeRSGroup(String groupName) throws IOException; void removeServersFromRSGroup(Set servers) throws IOException; void moveServersToRSGroup(Set servers, String targetGroup) throws IOException; void setRSGroup(Set tables, String groupName) throws IOException; boolean balanceRSGroup(String groupName) throws IOException; The shell commands for rs group are not changed. The main difference on the implementation is that, now the rs group for a table is stored in TableDescriptor, instead of in RSGroupInfo, so the getTables method of RSGroupInfo has been deprecated. And if you use the above Admin methods to get the RSGroupInfo, its getTables method will always return empty. Of course the behavior for the old RSGroupAdminEndpoint is not changed, we will fill the tables field of the RSGroupInfo before returning, to make it compatible with old hbase client/shell. When upgrading, the migration between the RSGroupInfo and TableDescriptor will be done automatically. It will take sometime, but it is fine to restart master in the middle, the migration will continue after restart. > Move rsgroup feature into core of HBase > --- > > Key: HBASE-22514 > URL: https://issues.apache.org/jira/browse/HBASE-22514 > Project: HBase > Issue Type: Umbrella > Components: Admin, Client, rsgroup >Reporter: Yechao Chen >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22514.master.001.patch, > image-2019-05-31-18-25-38-217.png > > > The class RSGroupAdminClient is not public > we need to use java api RSGroupAdminClient to manager RSG > so RSGroupAdminClient should be public > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23854) Documentation update of external_apis.adoc#example-scala-code
[
https://issues.apache.org/jira/browse/HBASE-23854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042575#comment-17042575
]
Hudson commented on HBASE-23854:
Results for branch master
[build #1639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1639/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Documentation update of external_apis.adoc#example-scala-code
> -
>
> Key: HBASE-23854
> URL: https://issues.apache.org/jira/browse/HBASE-23854
> Project: HBase
> Issue Type: Task
> Components: documentation
>Reporter: Michael Heil
>Assignee: Michael Heil
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: HBASE-23854.patch
>
>
> Update the Example Scala Code in the Reference Guide as it contains
> deprecated content such as
> * new HBaseConfiguration()
> * new HTable(conf, "mytable")
> * add(Bytes.toBytes("ids"),Bytes.toBytes("id1"),Bytes.toBytes("one"))
> Replace it with:
> * HBaseConfiguration.create()
> * TableName.valueOf({color:#6a8759}"mytable"{color})
> *
> addColumn(Bytes.toBytes({color:#6a8759}"ids"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"id1"{color}){color:#cc7832},{color}Bytes.toBytes({color:#6a8759}"one"{color}))
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23561) Look up of Region in Master by encoded region name is O(n)
[
https://issues.apache.org/jira/browse/HBASE-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042574#comment-17042574
]
Hudson commented on HBASE-23561:
Results for branch master
[build #1639 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1639/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1639//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Look up of Region in Master by encoded region name is O(n)
> --
>
> Key: HBASE-23561
> URL: https://issues.apache.org/jira/browse/HBASE-23561
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Michael Stack
>Assignee: Minwoo Kang
>Priority: Trivial
> Fix For: 3.0.0
>
>
> {{ public RegionState getRegionState(final String encodedRegionName) {
> // TODO: Need a map but it is just dispatch merge...
> for (RegionStateNode node: regionsMap.values()) {
> if (node.getRegionInfo().getEncodedName().equals(encodedRegionName)) {
> return node.toRegionState();
> }
> }
> return null;
> }}}
> It is not used much so making it trivial.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23561) Look up of Region in Master by encoded region name is O(n)
[
https://issues.apache.org/jira/browse/HBASE-23561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042567#comment-17042567
]
Duo Zhang commented on HBASE-23561:
---
This can be backported to all active branches?
> Look up of Region in Master by encoded region name is O(n)
> --
>
> Key: HBASE-23561
> URL: https://issues.apache.org/jira/browse/HBASE-23561
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Michael Stack
>Assignee: Minwoo Kang
>Priority: Trivial
> Fix For: 3.0.0
>
>
> {{ public RegionState getRegionState(final String encodedRegionName) {
> // TODO: Need a map but it is just dispatch merge...
> for (RegionStateNode node: regionsMap.values()) {
> if (node.getRegionInfo().getEncodedName().equals(encodedRegionName)) {
> return node.toRegionState();
> }
> }
> return null;
> }}}
> It is not used much so making it trivial.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22514) Move rsgroup feature into core of HBase
[ https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22514: -- Hadoop Flags: Incompatible change > Move rsgroup feature into core of HBase > --- > > Key: HBASE-22514 > URL: https://issues.apache.org/jira/browse/HBASE-22514 > Project: HBase > Issue Type: Umbrella > Components: Admin, Client, rsgroup >Reporter: Yechao Chen >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-22514.master.001.patch, > image-2019-05-31-18-25-38-217.png > > > The class RSGroupAdminClient is not public > we need to use java api RSGroupAdminClient to manager RSG > so RSGroupAdminClient should be public > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on issue #754: HBASE-22978 : Online slow response log
virajjasani commented on issue #754: HBASE-22978 : Online slow response log URL: https://github.com/apache/hbase/pull/754#issuecomment-589948906 > I think the patch looks in a decent shape after the refactor.. There are some minor comments I posted... > > Also, few things you mentioned you'd do in follow-on tasks.. > > * too-large responses > * metrics support > * thrift-api equivalent > > Please track them as sub-tasks so that we don't lose track of them... No worries, I am full aware of all follow up tasks and going to create JIRAs right after merging this. Thanks This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382894986
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382906513
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382895260
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382894446
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogRecorder.java
##
@@ -0,0 +1,153 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.BlockingWaitStrategy;
+import com.lmax.disruptor.RingBuffer;
+import com.lmax.disruptor.dsl.Disruptor;
+import com.lmax.disruptor.dsl.ProducerType;
+
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.util.Threads;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+import org.apache.hbase.thirdparty.com.google.common.base.Preconditions;
+
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Online SlowLog Provider Service that keeps slow RPC logs in the ring buffer.
+ * The service uses LMAX Disruptor to save slow records which are then
consumed by
+ * a queue and based on the ring buffer size, the available records are then
fetched
+ * from the queue in thread-safe manner.
+ */
+@InterfaceAudience.Private
+@InterfaceStability.Evolving
+public class SlowLogRecorder {
+
+ private final Disruptor disruptor;
+ private final SlowLogEventHandler slowLogEventHandler;
+ private final int eventCount;
+ private final boolean isOnlineSlowLogProviderEnabled;
+
+ private static final String SLOW_LOG_RING_BUFFER_SIZE =
+"hbase.regionserver.slowlog.ringbuffer.size";
+
+ /**
+ * Initialize disruptor with configurable ringbuffer size
+ */
+ public SlowLogRecorder(Configuration conf) {
+isOnlineSlowLogProviderEnabled =
conf.getBoolean(HConstants.SLOW_LOG_BUFFER_ENABLED_KEY,
+ HConstants.DEFAULT_ONLINE_LOG_PROVIDER_ENABLED);
+
+if (!isOnlineSlowLogProviderEnabled) {
+ this.disruptor = null;
Review comment:
They all are final and hence defined null by default. If ring buffer config
is not enabled, no point of initializing disruptor ring buffer right, same
reason. And yes we do want them final(just initialize them once)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382895009
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382895097
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
+ throws Exception {
+final RpcLogDetails rpcCallDetails = event.getPayload();
+final RpcCall rpcCall = rpcCallDetails.getRpcCall();
+final String clientAddress = rpcCallDetails.getClientAddress();
+final long responseSize = rpcCallDetails.getResponseSize();
+final String className = rpcCallDetails.getClassName();
+Descriptors.MethodDescriptor methodDescriptor = rpcCall.getMethod();
+Message param = rpcCall.getParam();
+long receiveTime = rpcCall.getReceiveTime();
+long startTime = rpcCall.getStartTime();
+long endTime = System.currentTimeMillis();
+int processingTime = (int) (endTime - startTime);
+int qTime = (int) (startTime - receiveTime);
+final SlowLogParams slowLogParams = ProtobufUtil.getSlowLogParams(param);
+int numGets = 0;
+int numMutations = 0;
+int numServiceCalls = 0;
+if (param instanceof ClientProtos.MultiRequest) {
+ ClientProtos.MultiRequest multi = (ClientProtos.MultiRequest) param;
+ for (ClientProtos.RegionAction regionAction :
multi.getRegionActionList()) {
+for (ClientProtos.Action action : regionAction.getActionList()) {
+ if (action.hasMutation()) {
+numMutations++;
+ }
+ if (action.hasGet()) {
+numGets++;
+ }
+ if (action.hasServiceCall()) {
+numServiceCalls++;
+ }
+}
+ }
+}
+final String userName =
rpcCall.getRequestUserName().orElse(StringUtils.EMPTY);
+final String methodDescriptorName =
+ methodDescriptor != null ? methodDescriptor.getName() :
StringUtils.EMPTY;
+SlowLogPayload slowLogPayload = SlowLogPayload.newBuilder()
+ .setCallDetails(methodDescriptorName
[GitHub] [hbase] virajjasani commented on a change in pull request #754: HBASE-22978 : Online slow response log
virajjasani commented on a change in pull request #754: HBASE-22978 : Online
slow response log
URL: https://github.com/apache/hbase/pull/754#discussion_r382895385
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/slowlog/SlowLogEventHandler.java
##
@@ -0,0 +1,205 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.regionserver.slowlog;
+
+import com.lmax.disruptor.EventHandler;
+import com.lmax.disruptor.RingBuffer;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Queue;
+import java.util.stream.Collectors;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.hbase.client.SlowLogParams;
+import org.apache.hadoop.hbase.ipc.RpcCall;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos;
+import org.apache.hbase.thirdparty.com.google.protobuf.Descriptors;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hbase.thirdparty.com.google.common.collect.EvictingQueue;
+import org.apache.hbase.thirdparty.com.google.common.collect.Queues;
+import org.apache.hbase.thirdparty.com.google.protobuf.Message;
+
+import org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil;
+import org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos;
+import
org.apache.hadoop.hbase.shaded.protobuf.generated.TooSlowLog.SlowLogPayload;
+
+/**
+ * Event Handler run by disruptor ringbuffer consumer
+ */
+@InterfaceAudience.Private
+class SlowLogEventHandler implements EventHandler {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(SlowLogEventHandler.class);
+
+ private final Queue queue;
+
+ SlowLogEventHandler(int eventCount) {
+EvictingQueue evictingQueue =
EvictingQueue.create(eventCount);
+queue = Queues.synchronizedQueue(evictingQueue);
+ }
+
+ /**
+ * Called when a publisher has published an event to the {@link RingBuffer}
+ *
+ * @param event published to the {@link RingBuffer}
+ * @param sequence of the event being processed
+ * @param endOfBatch flag to indicate if this is the last event in a batch
from
+ * the {@link RingBuffer}
+ * @throws Exception if the EventHandler would like the exception handled
further up the chain
+ */
+ @Override
+ public void onEvent(RingBufferEnvelope event, long sequence, boolean
endOfBatch)
Review comment:
That should be fine, not useful for our use-case because our consumers are
going to be multiple synchronized queues (as of now only one queue)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[
https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042509#comment-17042509
]
Guanghao Zhang commented on HBASE-23864:
{quote}This breaks TestSplitOrMergeStatus#testSplitSwitch in branch-2 (seems
fine on master)
{quote}
It's fine on master. Because the async client's admin.split() method not throw
the exception out. This should be another issue. Let me check that how to fix
this.
> No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when
> split/merge is disabled
> ---
>
> Key: HBASE-23864
> URL: https://issues.apache.org/jira/browse/HBASE-23864
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10
>
> Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG
>
>
> Now even the split/merge is disabled, master will submit a
> SplitTableRegionProcedure, too. And rollback it when execute failed. I
> thought the split/merge switch is a cluster level swtich. Master can check it
> early and no need to submit
> SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch
> is disabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23864: --- Status: Patch Available (was: Reopened) > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10 > > Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23864) No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge is disabled
[ https://issues.apache.org/jira/browse/HBASE-23864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-23864: --- Attachment: HBASE-23864-addendum.patch > No need to submit SplitTableRegionProcedure/MergeTableRegionsProcedure when > split/merge is disabled > --- > > Key: HBASE-23864 > URL: https://issues.apache.org/jira/browse/HBASE-23864 > Project: HBase > Issue Type: Improvement >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.4, 2.1.10 > > Attachments: HBASE-23864-addendum.patch, HBASE-23864.PNG > > > Now even the split/merge is disabled, master will submit a > SplitTableRegionProcedure, too. And rollback it when execute failed. I > thought the split/merge switch is a cluster level swtich. Master can check it > early and no need to submit > SplitTableRegionProcedure/MergeTableRegionsProcedure when split/merge switch > is disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] infraio merged pull request #1185: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg…
infraio merged pull request #1185: HBASE-23864 No need to submit SplitTableRegionProcedure/MergeTableReg… URL: https://github.com/apache/hbase/pull/1185 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
