[jira] [Commented] (HBASE-25837) TestRollingRestart is flaky
[
https://issues.apache.org/jira/browse/HBASE-25837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341175#comment-17341175
]
Hudson commented on HBASE-25837:
Results for branch master
[build #286 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286//console].
> TestRollingRestart is flaky
> ---
>
> Key: HBASE-25837
> URL: https://issues.apache.org/jira/browse/HBASE-25837
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Yulin Niu
>Assignee: Yulin Niu
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
> Attachments: screenshot-1.png
>
>
> the method *waitForRSShutdownToStartAndFinish* is incorrect, for the judgment
> method *sm.areDeadServersInProgress* may execute before scp is fired
> !screenshot-1.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341176#comment-17341176
]
Hudson commented on HBASE-25774:
Results for branch master
[build #286 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/286//console].
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.3, 2.3.5.1
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341124#comment-17341124 ] Michael Stack commented on HBASE-25032: --- I suggest we not do a 2.3.5.1 but a 2.3.6. [~apurtell] > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Caroline Zhou >Priority: Major > Labels: master, regionserver > Fix For: 3.0.0-alpha-1, 2.5.0 > > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834993503 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 1m 47s | root in master failed. | | -1 :x: | compile | 0m 42s | hbase-server in master failed. | | +1 :green_heart: | checkstyle | 1m 35s | master passed | | -1 :x: | spotbugs | 0m 36s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 49s | root in the patch failed. | | -1 :x: | compile | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | cc | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 40s | hbase-server in the patch failed. | | +1 :green_heart: | checkstyle | 1m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 1m 59s | The patch causes 16 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 4m 2s | The patch causes 16 errors with Hadoop v3.2.1. | | -1 :x: | hadoopcheck | 6m 1s | The patch causes 16 errors with Hadoop v3.3.0. | | -1 :x: | hbaseprotoc | 0m 37s | hbase-server in the patch failed. | | -1 :x: | spotbugs | 0m 34s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 32s | The patch does not generate ASF License warnings. | | | | 34m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 1d7f3a118457 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/branch-compile-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/branch-spotbugs-hbase-server.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | cc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-javac-3.3.0.txt | | hbaseprotoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-server.txt | | spotbugs |
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834989576 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 2m 20s | root in master failed. | | -1 :x: | compile | 0m 39s | hbase-server in master failed. | | -1 :x: | shadedjars | 6m 0s | branch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 15s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 20s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 8s | root in the patch failed. | | -1 :x: | compile | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 40s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 6m 9s | patch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 57s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 4s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 41s | hbase-server in the patch failed. | | | | 29m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1fa7f9218d83 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-11.0.10+9 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/branch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/testReport/ | | Max. process+thread count | 303 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834987575 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 1m 47s | root in master failed. | | -1 :x: | compile | 0m 39s | hbase-server in master failed. | | -1 :x: | shadedjars | 6m 3s | branch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 49s | root in the patch failed. | | -1 :x: | compile | 0m 38s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 38s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 6m 10s | patch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 44s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 45s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 38s | hbase-server in the patch failed. | | | | 27m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f8097813ecc4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/branch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/branch-shadedjars.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/testReport/ | | Max. process+thread count | 339 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834966081 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 1m 59s | root in master failed. | | -1 :x: | compile | 0m 44s | hbase-server in master failed. | | +1 :green_heart: | checkstyle | 1m 35s | master passed | | -1 :x: | spotbugs | 0m 38s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 47s | root in the patch failed. | | -1 :x: | compile | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | cc | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 40s | hbase-server in the patch failed. | | +1 :green_heart: | checkstyle | 1m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 2m 1s | The patch causes 16 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 4m 4s | The patch causes 16 errors with Hadoop v3.2.1. | | -1 :x: | hadoopcheck | 6m 10s | The patch causes 16 errors with Hadoop v3.3.0. | | -1 :x: | hbaseprotoc | 0m 37s | hbase-server in the patch failed. | | -1 :x: | spotbugs | 0m 31s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 31s | The patch does not generate ASF License warnings. | | | | 35m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 38e3e840b8a9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/branch-compile-hbase-server.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/branch-spotbugs-hbase-server.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | cc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-compile-hbase-server.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-javac-3.3.0.txt | | hbaseprotoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-general-check/output/patch-hbaseprotoc-hbase-server.txt | | spotbugs |
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834961724 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 2m 22s | root in master failed. | | -1 :x: | compile | 0m 41s | hbase-server in master failed. | | -1 :x: | shadedjars | 6m 11s | branch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 5s | root in the patch failed. | | -1 :x: | compile | 0m 41s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 41s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 6m 3s | patch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 57s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 4s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 41s | hbase-server in the patch failed. | | | | 29m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d034c6a3d124 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-11.0.10+9 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/branch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/branch-shadedjars.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/testReport/ | | Max. process+thread count | 275 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
Apache-HBase commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834960180 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 2m 1s | root in master failed. | | -1 :x: | compile | 0m 37s | hbase-server in master failed. | | -1 :x: | shadedjars | 6m 6s | branch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 46s | root in the patch failed. | | -1 :x: | compile | 0m 40s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 40s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 6m 7s | patch has 16 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 48s | hbase-common in the patch passed. | | -1 :x: | unit | 0m 39s | hbase-server in the patch failed. | | | | 27m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3244 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2944aeb6f8f8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 02b018cf1a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/branch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/branch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/branch-shadedjars.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/testReport/ | | Max. process+thread count | 340 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-common hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3244/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341095#comment-17341095
]
Andrew Kyle Purtell commented on HBASE-25774:
-
Yes, I marked the revert with HBASE-25774 . I think we need to set the fix
versions for it. I also updated fix versions on HBASE-25032. We are pulling the
2.3.5 release from the distribution mirrors once 2.3.5.1 is out, and 2.3.5.1
will have a correct change log, so I think we will be good. Please let me know
if you'd like to see something done differently.
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.3, 2.3.5.1
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell commented on pull request #3244: [WIP] HBASE-25869 WAL value compression
apurtell commented on pull request #3244: URL: https://github.com/apache/hbase/pull/3244#issuecomment-834951042 Clean up the commit message. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] apurtell commented on a change in pull request #3244: [WIP] HBASE-25869 WAL value compression
apurtell commented on a change in pull request #3244: URL: https://github.com/apache/hbase/pull/3244#discussion_r628663543 ## File path: hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java ## @@ -230,6 +230,11 @@ public static long getKeyDataStructureSize(int rlength, int flength, int qlength DeleteColumn((byte)12), DeleteFamily((byte)14), +// Effective maximum is 127 (Byte.MAX_VALUE). We set the high order bit of the +// type byte in the WAL codecs to indicate, in a backwards compatible way, if the +// value is compressed there. +EffectiveMaximum((byte)Byte.MAX_VALUE), Review comment: This is the one ugly thing about the trick I use to encode whether or not a value is compressed in a fully backwards compatible way. I'm not sure if anything need be done with `Maximum` below. It is no longer a valid value. Because we overload the high order bit the max KV type value is 127 after this change. Of course, as I am proposing this change, I think it is fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341093#comment-17341093
]
Duo Zhang commented on HBASE-25774:
---
Oh, good, just noticed that you committed the revert patch as HBASE-25774.
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.3, 2.3.5.1
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] apurtell commented on a change in pull request #3244: [WIP] HBASE-25869 WAL value compression
apurtell commented on a change in pull request #3244:
URL: https://github.com/apache/hbase/pull/3244#discussion_r628662332
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java
##
@@ -220,6 +223,8 @@ public ByteString compress(byte[] data, Enum dictIndex) {
}
}
+ static final int VALUE_COMPRESS_THRESHOLD = 100;
Review comment:
This is arbitrary and will be revisited. Will do some research on the
minimum useful plaintext size for Deflate and some testing with various types
of data.
I don't think it makes sense to make this configurable. Virtually no
operator will understand how to do better than what we choose now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-25869) WAL value compression
[ https://issues.apache.org/jira/browse/HBASE-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25869: Status: Patch Available (was: Open) > WAL value compression > - > > Key: HBASE-25869 > URL: https://issues.apache.org/jira/browse/HBASE-25869 > Project: HBase > Issue Type: Bug > Components: Operability, wal >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > WAL storage can be expensive, especially if the cell values represented in > the edits are large, consisting of blobs or significant lengths of text. Such > WALs might need to be kept around for a fairly long time to satisfy > replication constraints on a space limited (or space -contended) filesystem. > We have a custom dictionary compression scheme for cell metadata that is > engaged when WAL compression is enabled in site configuration. This is fine > for that application, where we can expect the universe of values (and their > lengths) in the custom dictionaries to be constrained. For arbitrary values > it is better to use Deflate compression, which is a complete LZ-class > algorithm suitable for arbitrary albeit compressible data, is reasonably > fast, certainly fast enough for WALs, compresses well, and is universally > available as part of the Java runtime. > With a trick that encodes whether or not the cell value is compressed in the > high order bit of the type byte, this can be done in a backwards compatible > manner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] apurtell opened a new pull request #3244: [WIP] HBASE-25869 WAL value compression
apurtell opened a new pull request #3244: URL: https://github.com/apache/hbase/pull/3244 WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space-contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values and their lengths in the custom dictionaries to be constrained. For arbitrary cell values it is better to use Deflate compression, which is which is a complete LZ-class algorithm suitable for arbitrary albeit compressible data, is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341091#comment-17341091
]
Duo Zhang commented on HBASE-25774:
---
We didn't commit anything to branch other than master, so should we resolve
this issue as fixed and set fix versions for all active branches? Not sure,
just asking...
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.3, 2.3.5.1
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Kyle Purtell resolved HBASE-25774.
-
Fix Version/s: 2.4.3
2.5.0
1.7.0
3.0.0-alpha-1
2.3.5.1
Resolution: Fixed
Resolving via revert of HBASE-25032
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.3.5.1, 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.4.3
>
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25869) WAL value compression
[ https://issues.apache.org/jira/browse/HBASE-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25869: Description: WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space -contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values (and their lengths) in the custom dictionaries to be constrained. For arbitrary values it is better to use Deflate compression, which is a complete LZ-class algorithm suitable for arbitrary albeit compressible data, is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. was: WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space -contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values (and their lengths) in the custom dictionaries to be constrained. For arbitrary values it is better to use Deflate compression, which is a complete LZ-class algorithm suitable for arbitrary albeit compressable data, is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. > WAL value compression > - > > Key: HBASE-25869 > URL: https://issues.apache.org/jira/browse/HBASE-25869 > Project: HBase > Issue Type: Bug > Components: Operability, wal >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > WAL storage can be expensive, especially if the cell values represented in > the edits are large, consisting of blobs or significant lengths of text. Such > WALs might need to be kept around for a fairly long time to satisfy > replication constraints on a space limited (or space -contended) filesystem. > We have a custom dictionary compression scheme for cell metadata that is > engaged when WAL compression is enabled in site configuration. This is fine > for that application, where we can expect the universe of values (and their > lengths) in the custom dictionaries to be constrained. For arbitrary values > it is better to use Deflate compression, which is a complete LZ-class > algorithm suitable for arbitrary albeit compressible data, is reasonably > fast, certainly fast enough for WALs, compresses well, and is universally > available as part of the Java runtime. > With a trick that encodes whether or not the cell value is compressed in the > high order bit of the type byte, this can be done in a backwards compatible > manner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25869) WAL value compression
[ https://issues.apache.org/jira/browse/HBASE-25869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25869: Description: WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space -contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values (and their lengths) in the custom dictionaries to be constrained. For arbitrary values it is better to use Deflate compression, which is a complete LZ-class algorithm suitable for arbitrary albeit compressable data, is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. was: WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space -contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values (and their lengths) in the custom dictionaries to be constrained. For arbitrary values it is better to use Deflate compression, which is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. > WAL value compression > - > > Key: HBASE-25869 > URL: https://issues.apache.org/jira/browse/HBASE-25869 > Project: HBase > Issue Type: Bug > Components: Operability, wal >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > WAL storage can be expensive, especially if the cell values represented in > the edits are large, consisting of blobs or significant lengths of text. Such > WALs might need to be kept around for a fairly long time to satisfy > replication constraints on a space limited (or space -contended) filesystem. > We have a custom dictionary compression scheme for cell metadata that is > engaged when WAL compression is enabled in site configuration. This is fine > for that application, where we can expect the universe of values (and their > lengths) in the custom dictionaries to be constrained. For arbitrary values > it is better to use Deflate compression, which is a complete LZ-class > algorithm suitable for arbitrary albeit compressable data, is reasonably > fast, certainly fast enough for WALs, compresses well, and is universally > available as part of the Java runtime. > With a trick that encodes whether or not the cell value is compressed in the > high order bit of the type byte, this can be done in a backwards compatible > manner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25869) WAL value compression
Andrew Kyle Purtell created HBASE-25869: --- Summary: WAL value compression Key: HBASE-25869 URL: https://issues.apache.org/jira/browse/HBASE-25869 Project: HBase Issue Type: Bug Components: Operability, wal Reporter: Andrew Kyle Purtell Assignee: Andrew Kyle Purtell Fix For: 3.0.0-alpha-1, 2.5.0 WAL storage can be expensive, especially if the cell values represented in the edits are large, consisting of blobs or significant lengths of text. Such WALs might need to be kept around for a fairly long time to satisfy replication constraints on a space limited (or space -contended) filesystem. We have a custom dictionary compression scheme for cell metadata that is engaged when WAL compression is enabled in site configuration. This is fine for that application, where we can expect the universe of values (and their lengths) in the custom dictionaries to be constrained. For arbitrary values it is better to use Deflate compression, which is reasonably fast, certainly fast enough for WALs, compresses well, and is universally available as part of the Java runtime. With a trick that encodes whether or not the cell value is compressed in the high order bit of the type byte, this can be done in a backwards compatible manner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3242: HBASE-25867 Extra doc around ITBLL
Apache-HBase commented on pull request #3242: URL: https://github.com/apache/hbase/pull/3242#issuecomment-834923345 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 14s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 43s | branch-2.4 passed | | +1 :green_heart: | compile | 1m 43s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 7m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 1m 45s | the patch passed | | +1 :green_heart: | javac | 1m 45s | the patch passed | | +1 :green_heart: | shadedjars | 7m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 195m 27s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 13s | hbase-it in the patch passed. | | | | 230m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3242 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 109bda4ff38b 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 4de3841074 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/testReport/ | | Max. process+thread count | 2841 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341088#comment-17341088 ] Andrew Kyle Purtell commented on HBASE-25032: - [~reidchan] Note revert from branch-1. > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Caroline Zhou >Priority: Major > Labels: master, regionserver > Fix For: 3.0.0-alpha-1, 2.5.0 > > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341087#comment-17341087 ] Andrew Kyle Purtell edited comment on HBASE-25032 at 5/8/21, 12:50 AM: --- We are reverting this change from master, branch-1, branch-2, branch-2.3, and branch-2.4 due to HBASE-25774. This change can go back in after the issues are addressed. This was released in 2.3.5 but 2.3.5 will be withdrawn and replaced with 2.3.5.1, which will contain just a revert of this commit. See discussion on HBASE-25774. was (Author: apurtell): We are reverting this change from master, branch-2, branch-2.3, and branch-2.4 due to HBASE-25774. This change can go back in after the issues are addressed. This was released in 2.3.5 but 2.3.5 will be withdrawn and replaced with 2.3.5.1, which will contain just a revert of this commit. See discussion on HBASE-25774. > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Caroline Zhou >Priority: Major > Labels: master, regionserver > Fix For: 3.0.0-alpha-1, 2.5.0 > > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell updated HBASE-25032: Fix Version/s: (was: 2.4.3) (was: 2.3.5) (was: 1.7.0) > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Caroline Zhou >Priority: Major > Labels: master, regionserver > Fix For: 3.0.0-alpha-1, 2.5.0 > > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (HBASE-25032) Wait for region server to become online before adding it to online servers in Master
[ https://issues.apache.org/jira/browse/HBASE-25032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reopened HBASE-25032: - We are reverting this change from master, branch-2, branch-2.3, and branch-2.4 due to HBASE-25774. This change can go back in after the issues are addressed. This was released in 2.3.5 but 2.3.5 will be withdrawn and replaced with 2.3.5.1, which will contain just a revert of this commit. See discussion on HBASE-25774. > Wait for region server to become online before adding it to online servers in > Master > > > Key: HBASE-25032 > URL: https://issues.apache.org/jira/browse/HBASE-25032 > Project: HBase > Issue Type: Bug >Reporter: Sandeep Guggilam >Assignee: Caroline Zhou >Priority: Major > Labels: master, regionserver > Fix For: 3.0.0-alpha-1, 1.7.0, 2.5.0, 2.3.5, 2.4.3 > > > As part of RS start up, RS reports for duty to Master . Master acknowledges > the request and adds it to the onlineServers list for further assigning any > regions to the RS > Once Master acknowledges the reportForDuty and sends back the response, RS > does a bunch of stuff like initializing replication sources etc before > becoming online. However, sometimes there could be an issue with initializing > replication sources when it is unable to connect to peer clusters because of > some kerberos configuration and there would be a delay of around 20 mins in > becoming online. > > Since master considers it online, it tries to assign regions and which fails > with ServerNotRunningYet exception, then the master tries to unassign which > again fails with the same exception leading the region to FAILED_CLOSE state. > > It would be good to have a check to see if the RS is ready to accept the > assignment requests before adding it to online servers list which would > account for any such delays as described above -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25774) ServerManager.getOnlineServer may miss some region servers when refreshing state in some procedure implementations
[
https://issues.apache.org/jira/browse/HBASE-25774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341086#comment-17341086
]
Andrew Kyle Purtell commented on HBASE-25774:
-
I am reverting HBASE-25032 from master, branch-2, branch-2.3, and branch-2.4
now and will make the 2.3.5.1 and 2.4.3 releases. Voting starts Monday.
> ServerManager.getOnlineServer may miss some region servers when refreshing
> state in some procedure implementations
> --
>
> Key: HBASE-25774
> URL: https://issues.apache.org/jira/browse/HBASE-25774
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Xiaolin Ha
>Assignee: Duo Zhang
>Priority: Critical
>
> [https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3025/9/testReport/org.apache.hadoop.hbase.replication/TestSyncReplicationStandbyKillRS/precommit_checks___yetus_jdk8_Hadoop3_checks__/]
> {code:java}
> ...[truncated 391170 chars]...
> 76d634:45149.replicationSource,1] regionserver.HRegionServer(2351): STOPPED:
> Unexpected exception in RS:2;ece3af76d634:45149.replicationSource,1
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> regionserver.HeapMemoryManager(218): Stopping
> 2021-04-11T11:14:40,268 INFO [MemStoreFlusher.0]
> regionserver.MemStoreFlusher$FlushHandler(384): MemStoreFlusher.0 exiting
> 2021-04-11T11:14:40,268 INFO [RS:2;ece3af76d634:45149]
> flush.RegionServerFlushTableProcedureManager(118): Stopping region server
> flush procedure manager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> snapshot.RegionServerSnapshotManager(136): Stopping
> RegionServerSnapshotManager abruptly.
> 2021-04-11T11:14:40,270 INFO [RS:2;ece3af76d634:45149]
> regionserver.HRegionServer(1146): aborting server
> ece3af76d634,45149,1618139661734
> 2021-04-11T11:14:40,272 ERROR
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.ReplicationSource(428): Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> currentPath=null
> java.lang.IllegalStateException: Source should be active.
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.initialize(ReplicationSource.java:547)
> ~[classes/:?]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
> 2021-04-11T11:14:40,272 DEBUG
> [ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245]
> regionserver.HRegionServer(2576): Abort already in progress. Ignoring the
> current request with reason: Unexpected exception in
> ReplicationExecutor-0.replicationSource,1-ece3af76d634,44745,1618139625245
> {code}
> Maybe it should use HBASE-24877 to avoid failure of the initialize of
> ReplicationSource.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3242: HBASE-25867 Extra doc around ITBLL
Apache-HBase commented on pull request #3242: URL: https://github.com/apache/hbase/pull/3242#issuecomment-834881953 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 36s | branch-2.4 passed | | +1 :green_heart: | compile | 1m 24s | branch-2.4 passed | | +1 :green_heart: | shadedjars | 6m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 56s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 17s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | shadedjars | 6m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 139m 23s | hbase-server in the patch passed. | | +1 :green_heart: | unit | 1m 20s | hbase-it in the patch passed. | | | | 168m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3242 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a5ea5c01f3bc 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 4de3841074 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/testReport/ | | Max. process+thread count | 4402 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
Apache-HBase commented on pull request #3243: URL: https://github.com/apache/hbase/pull/3243#issuecomment-834861687 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 42s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 51s | master passed | | +1 :green_heart: | compile | 9m 16s | master passed | | +1 :green_heart: | checkstyle | 2m 26s | master passed | | +1 :green_heart: | spotbugs | 13m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 14s | root in the patch failed. | | -1 :x: | compile | 0m 17s | root in the patch failed. | | -0 :warning: | javac | 0m 17s | root in the patch failed. | | -0 :warning: | checkstyle | 0m 9s | The patch fails to run checkstyle in root | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 5s | The patch has no ill-formed XML file. | | -1 :x: | hadoopcheck | 0m 13s | The patch causes 10 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 0m 24s | The patch causes 10 errors with Hadoop v3.2.1. | | -1 :x: | hadoopcheck | 0m 36s | The patch causes 10 errors with Hadoop v3.3.0. | | -1 :x: | spotbugs | 0m 17s | root in the patch failed. | | -1 :x: | spotbugs | 0m 11s | hbase-http in the patch failed. | | -1 :x: | spotbugs | 0m 8s | hbase-it in the patch failed. | | -1 :x: | spotbugs | 0m 9s | hbase-rest in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 36m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3243 | | Optional Tests | dupname asflicense javac hadoopcheck xml compile spotbugs hbaseanti checkstyle | | uname | Linux 7be6dd91b6ab 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-compile-root.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-compile-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/buildtool-patch-checkstyle-root.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-javac-3.3.0.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-spotbugs-root.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-http.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-it.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3243/1/artifact/yetus-general-check/output/patch-spotbugs-hbase-rest.txt | | Max. process+thread count | 126 (vs. ulimit of 3) | | modules | C: . hbase-http hbase-it hbase-rest U: . | | Console
[GitHub] [hbase] Apache-HBase commented on pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
Apache-HBase commented on pull request #3243: URL: https://github.com/apache/hbase/pull/3243#issuecomment-834854538 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25844) Fix Jersey for hbase-server processes
[
https://issues.apache.org/jira/browse/HBASE-25844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341063#comment-17341063
]
Nick Dimiduk commented on HBASE-25844:
--
PRs for each of the subtasks are posted. In summary: this is what I believe is
required in order to finish shading our dependency on Jersey and have a JSR
311/JSR 339/JSR 370 runtime that is completely isolated from what comes in via
Hadoop transitive dependencies. There may be some stray references to
javax.ws.rs-api jars floating around, will take another sweep. In the mean
time...
[~zhangduo], [~elserj], [~stack], [~busbey], [~apurtell]: Please take a look.
> Fix Jersey for hbase-server processes
> -
>
> Key: HBASE-25844
> URL: https://issues.apache.org/jira/browse/HBASE-25844
> Project: HBase
> Issue Type: Task
> Components: master, regionserver, thirdparty
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> I spent some time trying to use Jersey from within the Master and it's not
> working. To summarize, we have unshaded resources from both
> jersey-server-1.19 and jersey-server-2.32 on the hbase-server classpath.
> Jersey's initialization uses ServiceLoader to look up concrete implementation
> classes of {{javax.ws.rs}} classes at runtime. Because we do not shade
> {{javax.ws.rs}} in hbase-thirdparty-jersey, an attempt to use shaded
> jersey-2.x still results in loading unshaded jersey-1.x jars, leading to an
> error like this
> {noformat}
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:96)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:275)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> {noformat}
> We cannot override what version of these classes are loaded at runtime via
> Java property because Jersey's load order implementation checks system
> properties as a last resort, not first thing as is claimed by javadoc.
> So I can think of two solutions.
> # One is to shade {{javax.ws.rs}} in hbase-thirdparty-jersey. This would
> shade both the interfaces and the resource files that are referenced at
> runtime, allowing for an entirely isolated jersey container instantiate.
> # Another idea is to add a custom {{ClassLoader}} that is inserted before
> jersey is initialized. This would filter out resources that are "banned",
> allowing our desired implementation through.
> Between these, I think (1) is better, but I don't know what else might break.
> I've made an effort of both, but with neither approach can I get a jersey
> environment to response from my resource class... either because the solution
> is incomplete, or because I don't have the jersey environment configured
> properly.
> See also some discussion that happened over on Slack,
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1618857521051700
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25864) Use shaded javax.ws.rs package classes
[
https://issues.apache.org/jira/browse/HBASE-25864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25864:
-
Status: Patch Available (was: Open)
> Use shaded javax.ws.rs package classes
> --
>
> Key: HBASE-25864
> URL: https://issues.apache.org/jira/browse/HBASE-25864
> Project: HBase
> Issue Type: Sub-task
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> Update anywhere that we use Jersey to make use of the shaded {{javax.ws.rs}}
> package name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #3243: HBASE-25864 Use shaded javax.ws.rs package classes
ndimiduk opened a new pull request #3243: URL: https://github.com/apache/hbase/pull/3243 Adapt to the changes provided by apache/hbase-thirdparty#51 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-thirdparty] Apache-HBase commented on pull request #51: HBASE-25863 Shade javax.ws.rs package for use with shaded Jersey
Apache-HBase commented on pull request #51: URL: https://github.com/apache/hbase-thirdparty/pull/51#issuecomment-834822757 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 0m 34s | master passed | | +1 :green_heart: | compile | 0m 14s | master passed | | +1 :green_heart: | javadoc | 0m 10s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 5s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 0m 39s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 2s | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 4s | hbase-shaded-jersey in the patch passed. | | +1 :green_heart: | unit | 0m 5s | hbase-shaded-jackson-jaxrs-json-provider in the patch passed. | | +1 :green_heart: | unit | 0m 29s | root in the patch passed. | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 4m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/2/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-thirdparty/pull/51 | | Optional Tests | dupname asflicense javac javadoc unit xml compile | | uname | Linux ef33a63546a8 5.4.0-1047-aws #49~18.04.1-Ubuntu SMP Wed Apr 28 23:08:58 UTC 2021 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / 1d73a2e | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/2/testReport/ | | Max. process+thread count | 395 (vs. ulimit of 1000) | | modules | C: hbase-shaded-jersey hbase-shaded-jackson-jaxrs-json-provider . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/2/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25868) Shade jackson-jaxrs-json-provider for use with shaded jersey
[ https://issues.apache.org/jira/browse/HBASE-25868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341057#comment-17341057 ] Nick Dimiduk commented on HBASE-25868: -- This change goes hand-in-hand with HBASE-25863, so including the patch on the same PR. > Shade jackson-jaxrs-json-provider for use with shaded jersey > > > Key: HBASE-25868 > URL: https://issues.apache.org/jira/browse/HBASE-25868 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > The only pluggable feature of the this JSR spec that we take advantage of is > using jackson-jaxrs-json-provider to get Jackson entity serialization. > Rewrite this provider as well so that its use of the JSR API classes is > against our shaded packages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25868) Shade jackson-jaxrs-json-provider for use with shaded jersey
[ https://issues.apache.org/jira/browse/HBASE-25868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25868: - Status: Patch Available (was: Open) > Shade jackson-jaxrs-json-provider for use with shaded jersey > > > Key: HBASE-25868 > URL: https://issues.apache.org/jira/browse/HBASE-25868 > Project: HBase > Issue Type: Sub-task > Components: hbase-thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > > The only pluggable feature of the this JSR spec that we take advantage of is > using jackson-jaxrs-json-provider to get Jackson entity serialization. > Rewrite this provider as well so that its use of the JSR API classes is > against our shaded packages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25868) Shade jackson-jaxrs-json-provider for use with shaded jersey
Nick Dimiduk created HBASE-25868: Summary: Shade jackson-jaxrs-json-provider for use with shaded jersey Key: HBASE-25868 URL: https://issues.apache.org/jira/browse/HBASE-25868 Project: HBase Issue Type: Sub-task Components: hbase-thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk The only pluggable feature of the this JSR spec that we take advantage of is using jackson-jaxrs-json-provider to get Jackson entity serialization. Rewrite this provider as well so that its use of the JSR API classes is against our shaded packages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3242: HBASE-25867 Extra doc around ITBLL
Apache-HBase commented on pull request #3242: URL: https://github.com/apache/hbase/pull/3242#issuecomment-834812601 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.4 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | branch-2.4 passed | | +1 :green_heart: | compile | 3m 50s | branch-2.4 passed | | +1 :green_heart: | checkstyle | 1m 31s | branch-2.4 passed | | +1 :green_heart: | spotbugs | 2m 40s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 16s | the patch passed | | +1 :green_heart: | compile | 3m 46s | the patch passed | | -0 :warning: | javac | 0m 40s | hbase-it generated 1 new + 102 unchanged - 1 fixed = 103 total (was 103) | | +1 :green_heart: | checkstyle | 1m 12s | hbase-server: The patch generated 0 new + 200 unchanged - 1 fixed = 200 total (was 201) | | +1 :green_heart: | checkstyle | 0m 18s | hbase-it: The patch generated 0 new + 45 unchanged - 1 fixed = 45 total (was 46) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 19s | Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 49m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3242 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux c4c9c05303d4 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 4de3841074 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-it.txt | | Max. process+thread count | 96 (vs. ulimit of 12500) | | modules | C: hbase-server hbase-it U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3242/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3241: HBASE-25860 Add metric for successful wal roll requests.
Apache-HBase commented on pull request #3241: URL: https://github.com/apache/hbase/pull/3241#issuecomment-834798479 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 3s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +0 :ok: | mvndep | 2m 27s | Maven dependency ordering for branch | | -1 :x: | mvninstall | 8m 22s | root in branch-1 failed. | | +1 :green_heart: | compile | 1m 14s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 1m 17s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 2m 23s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 34s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 21s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 10s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 38s | branch-1 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 10s | root in the patch failed. | | +1 :green_heart: | compile | 1m 23s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 1m 23s | the patch passed | | +1 :green_heart: | compile | 1m 25s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 1m 25s | the patch passed | | +1 :green_heart: | checkstyle | 2m 25s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 27s | patch has no errors when building our shaded downstream artifacts. | | -1 :x: | hadoopcheck | 1m 52s | The patch causes 10 errors with Hadoop v2.8.5. | | -1 :x: | hadoopcheck | 4m 34s | The patch causes 10 errors with Hadoop v2.9.2. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 1m 15s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 4m 40s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 28s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 0m 37s | hbase-hadoop2-compat in the patch passed. | | -1 :x: | unit | 191m 54s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 247m 59s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.mapreduce.TestLoadIncrementalHFiles | | | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesUseSecurityEndPoint | | | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles | | | hadoop.hbase.namequeues.TestSlowLogAccessor | | | hadoop.hbase.master.TestWarmupRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3241/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3241 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 3af5a5fd6734 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3241/out/precommit/personality/provided.sh | | git revision | branch-1 / e2b5c20 | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3241/1/artifact/out/branch-mvninstall-root.txt | | mvninstall |
[GitHub] [hbase] saintstack opened a new pull request #3242: HBASE-25867 Extra doc around ITBLL
saintstack opened a new pull request #3242: URL: https://github.com/apache/hbase/pull/3242 Minor edits to a few log messages. Explain how the '-c' option works when passed to ChaosMonkeyRunner. Some added notes on ITBLL. Fix whacky 'R' and 'Not r' thing in Master (shows when you run ITBLL). In HRS, report hostname and port when it checks in (was debugging issue where Master and HRS had different notions of its hostname). Spare a dirty FNFException on startup if base dir not yet in place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3238: HBASE-25860 Add metric for successful wal roll requests.
Apache-HBase commented on pull request #3238: URL: https://github.com/apache/hbase/pull/3238#issuecomment-834781906 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 54s | master passed | | +1 :green_heart: | compile | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 9m 50s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 1m 37s | the patch passed | | +1 :green_heart: | javac | 1m 37s | the patch passed | | +1 :green_heart: | shadedjars | 9m 44s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 1s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 46s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 230m 20s | hbase-server in the patch passed. | | | | 270m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3238 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6bf33b3f7b4a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/testReport/ | | Max. process+thread count | 3344 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25867) Extra doc around ITBLL
Michael Stack created HBASE-25867: - Summary: Extra doc around ITBLL Key: HBASE-25867 URL: https://issues.apache.org/jira/browse/HBASE-25867 Project: HBase Issue Type: Bug Components: documentation Reporter: Michael Stack Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.4 Added some doc around ITBLL to explain stuff I had difficultly with. Minor items such as log message & javadoc edits and explaining how to pass configuration to the ChaosMonkeyRunner. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3238: HBASE-25860 Add metric for successful wal roll requests.
Apache-HBase commented on pull request #3238: URL: https://github.com/apache/hbase/pull/3238#issuecomment-834761407 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 56s | master passed | | +1 :green_heart: | compile | 1m 38s | master passed | | +1 :green_heart: | shadedjars | 9m 0s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 48s | the patch passed | | +1 :green_heart: | compile | 1m 38s | the patch passed | | +1 :green_heart: | javac | 1m 38s | the patch passed | | +1 :green_heart: | shadedjars | 8m 54s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 204m 50s | hbase-server in the patch passed. | | | | 242m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3238 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 58e03bdd3fc6 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/testReport/ | | Max. process+thread count | 3527 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25866) RESTServer should implement Hadoop's Tool interface
Nick Dimiduk created HBASE-25866:
Summary: RESTServer should implement Hadoop's Tool interface
Key: HBASE-25866
URL: https://issues.apache.org/jira/browse/HBASE-25866
Project: HBase
Issue Type: Task
Components: REST
Reporter: Nick Dimiduk
The class that provides the main entry point for the REST gateway appears to be
entirely hand-constructed. Instead, this should probably use the common
{{Tool}} support provided by Hadoop, so that it can do things like parse
command line configuration parameters. For example, it could extend from
{{AbstractHBaseTool}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341016#comment-17341016 ] Sean Busbey commented on HBASE-25855: - Yes I believe that is the correct thing to do. > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-thirdparty] Apache-HBase commented on pull request #51: HBASE-25863 Shade javax.ws.rs package for use with shaded Jersey
Apache-HBase commented on pull request #51: URL: https://github.com/apache/hbase-thirdparty/pull/51#issuecomment-834687560 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 1s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 42s | master passed | | +1 :green_heart: | compile | 0m 5s | master passed | | +1 :green_heart: | javadoc | 0m 6s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 10s | the patch passed | | +1 :green_heart: | compile | 0m 4s | the patch passed | | +1 :green_heart: | javac | 0m 4s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 0s | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 4s | hbase-shaded-jersey in the patch passed. | | +1 :green_heart: | asflicense | 0m 7s | The patch does not generate ASF License warnings. | | | | 2m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/1/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-thirdparty/pull/51 | | Optional Tests | dupname asflicense javac javadoc unit xml compile | | uname | Linux d409c4c0b297 5.4.0-1047-aws #49~18.04.1-Ubuntu SMP Wed Apr 28 23:08:58 UTC 2021 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / 1d73a2e | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/1/testReport/ | | Max. process+thread count | 39 (vs. ulimit of 1000) | | modules | C: hbase-shaded-jersey U: hbase-shaded-jersey | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-51/1/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25863) Shade javax.ws.rs package for use with shaded Jersey
[ https://issues.apache.org/jira/browse/HBASE-25863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25863: - Status: Patch Available (was: Open) > Shade javax.ws.rs package for use with shaded Jersey > > > Key: HBASE-25863 > URL: https://issues.apache.org/jira/browse/HBASE-25863 > Project: HBase > Issue Type: Sub-task > Components: thirdparty >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Major > Fix For: thirdparty-3.5.1 > > > From a comment on the parent ticket, > bq. Jersey is an implementation of the specifications that define that > package and its subcomponents. Thus, to use a shaded version of Jersey, one > must also shade usage of this package as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-thirdparty] ndimiduk commented on pull request #51: HBASE-25863 Shade javax.ws.rs package for use with shaded Jersey
ndimiduk commented on pull request #51: URL: https://github.com/apache/hbase-thirdparty/pull/51#issuecomment-834686023 Parking this here for now. Will be back with a PR vs. hbase-core that makes use of this, so we can verify it's all working the way we intend. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase-thirdparty] ndimiduk opened a new pull request #51: HBASE-25863 Shade javax.ws.rs package for use with shaded Jersey
ndimiduk opened a new pull request #51: URL: https://github.com/apache/hbase-thirdparty/pull/51 From the [About](https://eclipse-ee4j.github.io/jersey/) text, > Jersey RESTful Web Services 2.x framework is open source, production quality, framework for > developing RESTful Web Services in Java that provides support for JAX-RS APIs and serves as a > JAX-RS (JSR 311 & JSR 339 & JSR 370) Reference Implementation. `javax.ws.rs` is defined by the JSRs, so it doesn't make sense that we could have multiple implementations of that JSR on the classpath simultaniously (via jersey-server-1.x and jersey-server-2.x jars) without them colliding. By shading over the JSR package space in the jersey-server-2.x implementation jars, we achieve a completely isolated JSR runtime. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25865:
-
Component/s: Usability
Operability
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master, Operability, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340995#comment-17340995
]
Nick Dimiduk commented on HBASE-25865:
--
My work on HBASE-25844 was motivated by this project, getting
{{ClusterMetrics}} out to the browser without contributing to our manual
servlet mess.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340992#comment-17340992
]
Nick Dimiduk commented on HBASE-25865:
--
The original scripts were written in Python, used Pandas to manipulate the raw
data, and Plot.ly to render charts in a Jupiter notebook. I've started a PoC
implementation using Vega-Lite to directly parse a json conversion of the
{{ClusterMetrics}} object and create an interactive chart. Of note, Vega seems
to handle the data volume produced by this cluster much better than Plot.ly did.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
`length` expects the key length or the length of the whole key-value pair.
You'll see that `KeyValue` allocates a byte array of length `length`(line 2266)
and so I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructor](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this is zero
as mentioned before. The
[check](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L537)
finally results into the exception: `Error:
java.lang.IllegalArgumentException: Invalid key length in KeyValue.
keyLength=0, KeyValueBytesHex=\x00\x00\x00\x00\x00Lc30f5d93[...], offset=0,
length=97`
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Work started] (HBASE-25865) Visualize current state of region assignment
[
https://issues.apache.org/jira/browse/HBASE-25865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-25865 started by Nick Dimiduk.
> Visualize current state of region assignment
>
>
> Key: HBASE-25865
> URL: https://issues.apache.org/jira/browse/HBASE-25865
> Project: HBase
> Issue Type: New Feature
> Components: master
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
>
> After several months of debugging and tuning the balancer and normalizer on a
> large production cluster, we found that working from visualizations of the
> current region state was very useful for understanding behaviors and
> quantifying improvements we made along the way. Specifically, we found that a
> chart of total assigned region count and total assigned region store files
> size per table per host was immensely useful for tuning the balancer.
> Histograms of store file size made understanding normalizer activity much
> more intuitive.
> Our scripts would parse the output of the shell's {{status 'detailed'}}
> command, extract the desired metric, and produce charts. I'd like to build
> into the master UI the equivalent functionality, with data coming directly
> from the {{ClusterMetrics object}}, and data rendered into an interactive
> chart rendered in the browser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25865) Visualize current state of region assignment
Nick Dimiduk created HBASE-25865:
Summary: Visualize current state of region assignment
Key: HBASE-25865
URL: https://issues.apache.org/jira/browse/HBASE-25865
Project: HBase
Issue Type: New Feature
Components: master
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
After several months of debugging and tuning the balancer and normalizer on a
large production cluster, we found that working from visualizations of the
current region state was very useful for understanding behaviors and
quantifying improvements we made along the way. Specifically, we found that a
chart of total assigned region count and total assigned region store files size
per table per host was immensely useful for tuning the balancer. Histograms of
store file size made understanding normalizer activity much more intuitive.
Our scripts would parse the output of the shell's {{status 'detailed'}}
command, extract the desired metric, and produce charts. I'd like to build into
the master UI the equivalent functionality, with data coming directly from the
{{ClusterMetrics object}}, and data rendered into an interactive chart rendered
in the browser.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25864) Use shaded javax.ws.rs package classes
Nick Dimiduk created HBASE-25864:
Summary: Use shaded javax.ws.rs package classes
Key: HBASE-25864
URL: https://issues.apache.org/jira/browse/HBASE-25864
Project: HBase
Issue Type: Sub-task
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Update anywhere that we use Jersey to make use of the shaded {{javax.ws.rs}}
package name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-25844) Fix Jersey for hbase-server processes
[
https://issues.apache.org/jira/browse/HBASE-25844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-25844:
-
Component/s: thirdparty
> Fix Jersey for hbase-server processes
> -
>
> Key: HBASE-25844
> URL: https://issues.apache.org/jira/browse/HBASE-25844
> Project: HBase
> Issue Type: Task
> Components: master, regionserver, thirdparty
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> I spent some time trying to use Jersey from within the Master and it's not
> working. To summarize, we have unshaded resources from both
> jersey-server-1.19 and jersey-server-2.32 on the hbase-server classpath.
> Jersey's initialization uses ServiceLoader to look up concrete implementation
> classes of {{javax.ws.rs}} classes at runtime. Because we do not shade
> {{javax.ws.rs}} in hbase-thirdparty-jersey, an attempt to use shaded
> jersey-2.x still results in loading unshaded jersey-1.x jars, leading to an
> error like this
> {noformat}
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:96)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:275)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> {noformat}
> We cannot override what version of these classes are loaded at runtime via
> Java property because Jersey's load order implementation checks system
> properties as a last resort, not first thing as is claimed by javadoc.
> So I can think of two solutions.
> # One is to shade {{javax.ws.rs}} in hbase-thirdparty-jersey. This would
> shade both the interfaces and the resource files that are referenced at
> runtime, allowing for an entirely isolated jersey container instantiate.
> # Another idea is to add a custom {{ClassLoader}} that is inserted before
> jersey is initialized. This would filter out resources that are "banned",
> allowing our desired implementation through.
> Between these, I think (1) is better, but I don't know what else might break.
> I've made an effort of both, but with neither approach can I get a jersey
> environment to response from my resource class... either because the solution
> is incomplete, or because I don't have the jersey environment configured
> properly.
> See also some discussion that happened over on Slack,
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1618857521051700
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25863) Shade javax.ws.rs package for use with shaded Jersey
Nick Dimiduk created HBASE-25863: Summary: Shade javax.ws.rs package for use with shaded Jersey Key: HBASE-25863 URL: https://issues.apache.org/jira/browse/HBASE-25863 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Nick Dimiduk Assignee: Nick Dimiduk Fix For: thirdparty-3.5.1 >From a comment on the parent ticket, bq. Jersey is an implementation of the specifications that define that package and its subcomponents. Thus, to use a shaded version of Jersey, one must also shade usage of this package as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25844) Fix Jersey for hbase-server processes
[
https://issues.apache.org/jira/browse/HBASE-25844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340981#comment-17340981
]
Nick Dimiduk commented on HBASE-25844:
--
I have a working Jersey setup now, by way of shading the {{javax.ws.rs}}
package prefix. I think this is the correct approach because Jersey is an
implementation of the specifications that define that package and its
subcomponents. Thus, to use a shaded version of Jersey, one must also shade
usage of this package as well.
> Fix Jersey for hbase-server processes
> -
>
> Key: HBASE-25844
> URL: https://issues.apache.org/jira/browse/HBASE-25844
> Project: HBase
> Issue Type: Task
> Components: master, regionserver
>Affects Versions: 3.0.0-alpha-1, 2.5.0
>Reporter: Nick Dimiduk
>Priority: Major
>
> I spent some time trying to use Jersey from within the Master and it's not
> working. To summarize, we have unshaded resources from both
> jersey-server-1.19 and jersey-server-2.32 on the hbase-server classpath.
> Jersey's initialization uses ServiceLoader to look up concrete implementation
> classes of {{javax.ws.rs}} classes at runtime. Because we do not shade
> {{javax.ws.rs}} in hbase-thirdparty-jersey, an attempt to use shaded
> jersey-2.x still results in loading unshaded jersey-1.x jars, leading to an
> error like this
> {noformat}
> java.lang.AbstractMethodError:
> javax.ws.rs.core.UriBuilder.uri(Ljava/lang/String;)Ljavax/ws/rs/core/UriBuilder;
> at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:96)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:275)
> at
> org.apache.hbase.thirdparty.org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
> at
> org.apache.hbase.thirdparty.org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> {noformat}
> We cannot override what version of these classes are loaded at runtime via
> Java property because Jersey's load order implementation checks system
> properties as a last resort, not first thing as is claimed by javadoc.
> So I can think of two solutions.
> # One is to shade {{javax.ws.rs}} in hbase-thirdparty-jersey. This would
> shade both the interfaces and the resource files that are referenced at
> runtime, allowing for an entirely isolated jersey container instantiate.
> # Another idea is to add a custom {{ClassLoader}} that is inserted before
> jersey is initialized. This would filter out resources that are "banned",
> allowing our desired implementation through.
> Between these, I think (1) is better, but I don't know what else might break.
> I've made an effort of both, but with neither approach can I get a jersey
> environment to response from my resource class... either because the solution
> is incomplete, or because I don't have the jersey environment configured
> properly.
> See also some discussion that happened over on Slack,
> https://apache-hbase.slack.com/archives/C13K8NVAM/p1618857521051700
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25851) Make LoadBalancer not extend Configurable interface
[
https://issues.apache.org/jira/browse/HBASE-25851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340979#comment-17340979
]
Hudson commented on HBASE-25851:
Results for branch branch-2
[build #244 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make LoadBalancer not extend Configurable interface
> ---
>
> Key: HBASE-25851
> URL: https://issues.apache.org/jira/browse/HBASE-25851
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> On master, LoadBalancer has a setClusterInfoProvider method and on branch-2,
> LoadBalancer has a setMasterServices method. Both ClusterInfoProvider and
> MasterServices could give a Configuration out so I do not think we need to
> expose another setConf method.
> When refactoring StochasticLoadBalancer, the initialization order is also a
> pain as after setConf, we will load a lot of configuration but at that time,
> the ClusterInfoProvider is not present, and easy to cause NPE...
> And it could also help simplity locking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25854) Remove redundant AM in-memory state changes in CatalogJanitor
[
https://issues.apache.org/jira/browse/HBASE-25854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340980#comment-17340980
]
Hudson commented on HBASE-25854:
Results for branch branch-2
[build #244 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/244/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove redundant AM in-memory state changes in CatalogJanitor
> -
>
> Key: HBASE-25854
> URL: https://issues.apache.org/jira/browse/HBASE-25854
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> In CatalogJanitor we schedule GCRegionProcedure to clean up both filesystem
> and in-memory state after a split, and GCMultipleMergedRegionsProcedure to do
> the same for merges.
> Both GCRegionProcedure and GCMultipleMergedRegionsProcedure clean up
> in-memory state, but CatalogJanitor also does this redundantly just after
> scheduling the procedures.
> The cleanup should be done in only one place. Presumably we are using the
> procedures to do it in a principled way. This is least a nit, but probably a
> source of future bugs. Remove the redundancy in CatalogJanitor and fix any
> follow on issues, like test failures.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340976#comment-17340976 ] Nick Dimiduk commented on HBASE-25855: -- So i believe this would be a backwards-incompatible change, in that anything in downstream projects that imports a jersey class will need to modify their package import. With this change, should I bump the hbase-thirdparty version to 4.0.0-SNAPSHOT? > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340975#comment-17340975 ] Nick Dimiduk commented on HBASE-25855: -- Got it, thanks [~psomogyi] ! > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] shahrs87 opened a new pull request #3241: HBASE-25860 Add metric for successful wal roll requests.
shahrs87 opened a new pull request #3241: URL: https://github.com/apache/hbase/pull/3241 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3238: HBASE-25860 Add metric for successful wal roll requests.
Apache-HBase commented on pull request #3238: URL: https://github.com/apache/hbase/pull/3238#issuecomment-834640835 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 54s | master passed | | +1 :green_heart: | compile | 3m 42s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | spotbugs | 2m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 38s | the patch passed | | +1 :green_heart: | compile | 3m 43s | the patch passed | | +1 :green_heart: | javac | 3m 43s | the patch passed | | +1 :green_heart: | checkstyle | 1m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 18m 11s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 54s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 50m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3238 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6aa47c4fc501 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3238/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340941#comment-17340941 ] Peter Somogyi commented on HBASE-25855: --- For hbase-thirdparty development is only done on the master branch. Both [3.4.0|https://lists.apache.org/thread.html/r033a0833ec97fde0a8381295134d7be4616545b599c67c7babe01ada%40%3Cdev.hbase.apache.org%3E] and [3.4.1|https://lists.apache.org/thread.html/r269bf61cdff2632407fed4774503a514ea0f24789d5d8c100b004962%40%3Cdev.hbase.apache.org%3E] were released last year. Jira versions weren't updated. I've corrected it now. I also archived hbase-thirdparty-3.4.2 version because there won't be a release with this version. > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340935#comment-17340935 ] Nick Dimiduk commented on HBASE-25855: -- [~zhangduo], [~busbey], [~psomogyi], [~stack] pardon me, I'm slightly unfamiliar with the developer hygiene related to this repository. I've committed this change to the master branch, which currently sits at 3.5.1-SNAPSHOT. From Jira, I see that 3.4.0 and 3.4.1 are not marked as released (though I see release tags in the master branch history). Do I need to check some other branches for backporting? Thanks. > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25855) Fix typo in jersey relocation path
[ https://issues.apache.org/jira/browse/HBASE-25855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-25855: - Fix Version/s: thirdparty-3.5.1 > Fix typo in jersey relocation path > -- > > Key: HBASE-25855 > URL: https://issues.apache.org/jira/browse/HBASE-25855 > Project: HBase > Issue Type: Task > Components: hbase-thirdparty >Affects Versions: thirdparty-3.4.1 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Minor > Fix For: thirdparty-3.5.1 > > > We shade to "jersery" instead of "jersey". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase-thirdparty] ndimiduk merged pull request #50: HBASE-25855 Fix typo in jersey relocation path
ndimiduk merged pull request #50: URL: https://github.com/apache/hbase-thirdparty/pull/50 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on a change in pull request #3238: HBASE-25860 Add metric for successful wal roll requests.
shahrs87 commented on a change in pull request #3238:
URL: https://github.com/apache/hbase/pull/3238#discussion_r628360727
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWAL.java
##
@@ -91,4 +92,12 @@ public void
logRollRequested(WALActionsListener.RollRequestReason reason) {
break;
}
}
+
+ public void postLogRoll(Path oldPath, Path newPath) throws IOException {
Review comment:
Thank you for the review. Fixed in latest commit.
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWAL.java
##
@@ -129,4 +136,25 @@ public void testPerTableWALMetrics() throws Exception {
assertEquals(i * numIters, tableAppendSize);
}
}
+
+ @Test
+ public void testLogRolls() throws IOException {
Review comment:
Thank you for the review. Fixed in latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase-thirdparty] Apache-HBase commented on pull request #50: HBASE-25855 Fix typo in jersey relocation path
Apache-HBase commented on pull request #50: URL: https://github.com/apache/hbase-thirdparty/pull/50#issuecomment-834606086 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 44s | master passed | | +1 :green_heart: | compile | 0m 5s | master passed | | +1 :green_heart: | javadoc | 0m 7s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 9s | the patch passed | | +1 :green_heart: | compile | 0m 5s | the patch passed | | +1 :green_heart: | javac | 0m 5s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 0s | The patch has no ill-formed XML file. | | +1 :green_heart: | javadoc | 0m 5s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 4s | hbase-shaded-jersey in the patch passed. | | +1 :green_heart: | asflicense | 0m 7s | The patch does not generate ASF License warnings. | | | | 2m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-50/2/artifact/yetus-precommit-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase-thirdparty/pull/50 | | Optional Tests | dupname asflicense javac javadoc unit xml compile | | uname | Linux e8576b3fc1bf 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux | | Build tool | maven | | git revision | master / ccc49e6 | | Default Java | Oracle Corporation-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-50/2/testReport/ | | Max. process+thread count | 39 (vs. ulimit of 1000) | | modules | C: hbase-shaded-jersey U: hbase-shaded-jersey | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-Thirdparty-PreCommit/job/PR-50/2/console | | versions | git=2.20.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-25862) Improve the Configuration File Descriptions accuracy
[ https://issues.apache.org/jira/browse/HBASE-25862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25862 started by Che Xun. --- > Improve the Configuration File Descriptions accuracy > > > Key: HBASE-25862 > URL: https://issues.apache.org/jira/browse/HBASE-25862 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Che Xun >Assignee: Che Xun >Priority: Minor > Fix For: 3.0.0-alpha-1 > > Attachments: image-2021-05-07-23-12-11-934.png > > > In Apache HBase ™ Reference Guide > *Need to modify part1：* > 7.Default Configuration ->7.4. log4j.properties > *explanation：* > we need to update to 7.4 log4j2.xml > *Need to modify part2：* > In the first paragraph of the chapter 8.1，the details list is not displayed > correctly，as show below： > > *!image-2021-05-07-23-12-11-934.png!* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25862) Improve the Configuration File Descriptions accuracy
[ https://issues.apache.org/jira/browse/HBASE-25862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340905#comment-17340905 ] Che Xun commented on HBASE-25862: - Hi, [~zhangduo], [~ndimiduk] Could you please review this PR and provide feedback? Recently I am reading the refrence guide, what kind of frequency to commit the issues is appropriate? Is it good to accumulate a bunch of issues to commit in one time, or establish a JIRA to record issue every time when I find? Will this cause a lot of similar commit? Thanks, please give me some suggestions. > Improve the Configuration File Descriptions accuracy > > > Key: HBASE-25862 > URL: https://issues.apache.org/jira/browse/HBASE-25862 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 3.0.0-alpha-1 >Reporter: Che Xun >Assignee: Che Xun >Priority: Minor > Fix For: 3.0.0-alpha-1 > > Attachments: image-2021-05-07-23-12-11-934.png > > > In Apache HBase ™ Reference Guide > *Need to modify part1：* > 7.Default Configuration ->7.4. log4j.properties > *explanation：* > we need to update to 7.4 log4j2.xml > *Need to modify part2：* > In the first paragraph of the chapter 8.1，the details list is not displayed > correctly，as show below： > > *!image-2021-05-07-23-12-11-934.png!* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3240: HBASE-25862 Improve the documentation of configration
Apache-HBase commented on pull request #3240: URL: https://github.com/apache/hbase/pull/3240#issuecomment-834560361 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 18s | master passed | | +0 :ok: | refguide | 3m 46s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 58s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 3m 39s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 18m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3240 | | Optional Tests | dupname asflicense refguide | | uname | Linux 60fddcc9c1f1 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/artifact/yetus-general-check/output/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 65 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25612) HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
[
https://issues.apache.org/jira/browse/HBASE-25612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340899#comment-17340899
]
Hudson commented on HBASE-25612:
Results for branch branch-1
[build #123 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/123/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/123//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/123//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/123//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> HMaster should abort if ReplicationLogCleaner is not able to delete oldWALs.
>
>
> Key: HBASE-25612
> URL: https://issues.apache.org/jira/browse/HBASE-25612
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.6.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Major
> Fix For: 1.7.0
>
>
> In our production cluster, we encountered an issue where the number of files
> within /hbase/oldWALs directory were growing exponentially from about 4000
> baseline to 15 and growing at the rate of 333 files per minute.
> On further investigation we found that ReplicatonLogCleaner thread was
> getting aborted since it was not able to talk to zookeeper. Stack trace below
> {noformat}
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729] zookeeper.ZKUtil -
> replicationLogCleaner-0x302e05e0d8f,
> quorum=zookeeper-0:2181,zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181,zookeeper-4:2181,
> baseZNode=/hbase Unable to get data of znode /hbase/replication/rs
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1229)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getData(RecoverableZooKeeper.java:374)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataNoWatch(ZKUtil.java:713)
> at
> org.apache.hadoop.hbase.replication.ReplicationQueuesClientZKImpl.getQueuesZNodeCversion(ReplicationQueuesClientZKImpl.java:87)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.loadWALsFromQueues(ReplicationLogCleaner.java:99)
> at
> org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.getDeletableFiles(ReplicationLogCleaner.java:70)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.checkAndDeleteFiles(CleanerChore.java:262)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$200(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:413)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$3.act(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:481)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:410)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore.access$100(CleanerChore.java:52)
> at
> org.apache.hadoop.hbase.master.cleaner.CleanerChore$1.run(CleanerChore.java:220)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> 2021-02-25 23:05:01,149 DEBUG [an-pool3-thread-1729]
> master.ReplicationLogCleaner -
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/replication/rs
> 2021-02-25 23:05:01,150 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - Failed to read zookeeper, skipping checking
> deletable files
> {noformat}
>
> {quote} 2021-02-25 23:05:01,149 WARN [an-pool3-thread-1729]
> master.ReplicationLogCleaner - ReplicationLogCleaner received abort,
> ignoring. Reason: Failed to get stat of replication rs node
> {quote}
>
> This line is more scary
[GitHub] [hbase] Apache-HBase commented on pull request #3240: HBASE-25862 Improve the documentation of configration
Apache-HBase commented on pull request #3240: URL: https://github.com/apache/hbase/pull/3240#issuecomment-834545887 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 8m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3240 | | Optional Tests | | | uname | Linux cfce9870c75c 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Max. process+thread count | 51 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3240: HBASE-25862 Improve the documentation of configration
Apache-HBase commented on pull request #3240: URL: https://github.com/apache/hbase/pull/3240#issuecomment-834539205 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | ||| _ Other Tests _ | | | | 2m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3240 | | Optional Tests | | | uname | Linux 22acb3e426fa 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6cfff27465 | | Max. process+thread count | 42 (vs. ulimit of 3) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3240/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] chexunn opened a new pull request #3240: HBASE-25862 Improve the documentation of configration
chexunn opened a new pull request #3240: URL: https://github.com/apache/hbase/pull/3240 Improve the documentation of configration section of reference guide -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25862) Improve the Configuration File Descriptions accuracy
Che Xun created HBASE-25862: --- Summary: Improve the Configuration File Descriptions accuracy Key: HBASE-25862 URL: https://issues.apache.org/jira/browse/HBASE-25862 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 3.0.0-alpha-1 Reporter: Che Xun Assignee: Che Xun Fix For: 3.0.0-alpha-1 Attachments: image-2021-05-07-23-12-11-934.png In Apache HBase ™ Reference Guide *Need to modify part1：* 7.Default Configuration ->7.4. log4j.properties *explanation：* we need to update to 7.4 log4j2.xml *Need to modify part2：* In the first paragraph of the chapter 8.1，the details list is not displayed correctly，as show below： *!image-2021-05-07-23-12-11-934.png!* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #3238: HBASE-25860 Add metric for successful wal roll requests.
virajjasani commented on a change in pull request #3238:
URL: https://github.com/apache/hbase/pull/3238#discussion_r628242734
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/MetricsWAL.java
##
@@ -91,4 +92,12 @@ public void
logRollRequested(WALActionsListener.RollRequestReason reason) {
break;
}
}
+
+ public void postLogRoll(Path oldPath, Path newPath) throws IOException {
Review comment:
nit: remove `throws IOException`?
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestMetricsWAL.java
##
@@ -129,4 +136,25 @@ public void testPerTableWALMetrics() throws Exception {
assertEquals(i * numIters, tableAppendSize);
}
}
+
+ @Test
+ public void testLogRolls() throws IOException {
Review comment:
same here, IOException is not thrown.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25851) Make LoadBalancer not extend Configurable interface
[
https://issues.apache.org/jira/browse/HBASE-25851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340841#comment-17340841
]
Hudson commented on HBASE-25851:
Results for branch master
[build #285 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Make LoadBalancer not extend Configurable interface
> ---
>
> Key: HBASE-25851
> URL: https://issues.apache.org/jira/browse/HBASE-25851
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> On master, LoadBalancer has a setClusterInfoProvider method and on branch-2,
> LoadBalancer has a setMasterServices method. Both ClusterInfoProvider and
> MasterServices could give a Configuration out so I do not think we need to
> expose another setConf method.
> When refactoring StochasticLoadBalancer, the initialization order is also a
> pain as after setConf, we will load a lot of configuration but at that time,
> the ClusterInfoProvider is not present, and easy to cause NPE...
> And it could also help simplity locking.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25854) Remove redundant AM in-memory state changes in CatalogJanitor
[
https://issues.apache.org/jira/browse/HBASE-25854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340842#comment-17340842
]
Hudson commented on HBASE-25854:
Results for branch master
[build #285 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/285/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove redundant AM in-memory state changes in CatalogJanitor
> -
>
> Key: HBASE-25854
> URL: https://issues.apache.org/jira/browse/HBASE-25854
> Project: HBase
> Issue Type: Sub-task
>Reporter: Andrew Kyle Purtell
>Assignee: Andrew Kyle Purtell
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3
>
>
> In CatalogJanitor we schedule GCRegionProcedure to clean up both filesystem
> and in-memory state after a split, and GCMultipleMergedRegionsProcedure to do
> the same for merges.
> Both GCRegionProcedure and GCMultipleMergedRegionsProcedure clean up
> in-memory state, but CatalogJanitor also does this redundantly just after
> scheduling the procedures.
> The cleanup should be done in only one place. Presumably we are using the
> procedures to do it in a principled way. This is least a nit, but probably a
> source of future bugs. Remove the redundancy in CatalogJanitor and fix any
> follow on issues, like test failures.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
`length` expects the key length or the length of the whole key-value pair.
You'll see that `KeyValue` allocates a byte array of length `length`(line 2266)
and so I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructor](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this zero as
mentioned before. The
[check](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L537)
finally results into the exception: `Error:
java.lang.IllegalArgumentException: Invalid key length in KeyValue.
keyLength=0, KeyValueBytesHex=\x00\x00\x00\x00\x00Lc30f5d93[...], offset=0,
length=97`
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
`length` expects the key length or the length of the whole key-value pair.
You'll see that `KeyValue` allocates a byte array of length `length`(line 2266)
and so I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructor](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this '0' as
mentioned before which in turn results into the exception: `Error:
java.lang.IllegalArgumentException: Invalid key length in KeyValue.
keyLength=0, KeyValueBytesHex=\x00\x00\x00\x00\x00Lc30f5d93[...], offset=0,
length=97`
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
`length` expects the key length or the length of the whole key-value pair.
You'll see that `KeyValue` allocates a byte array of length `length`(line 2266)
and so I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructor](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this '0' as
mentioned before.
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
`length` expects the key length or the length of the whole key-value pair.
You'll see that `KeyValue` allocates a byte array of length `length`(line 2266)
and so I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructure](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this '0' as
mentioned before.
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] offermannu commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
offermannu commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628175203
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
I tried this, but it doesn't work unfortunately. If you look at the
`readFields` method you'll see that it calls `KeyValue.create(DataInput in)`
which in turn calls [`KeyValue.create(int length, DataInput
in)`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L2281).
The length is read as integer from the DataInput which kind of matches what
`write` has written before in line 138 - so far so good (I'm refering to
original version btw).
Honestly I'm a little bit confused at this point and don't know whether
length expects the key length or the length of the whole key-value pair. You'll
see that `KeyValue` allocates a byte array of length `length`(line 2266) and so
I guess that it must be the length of key+value.
Now keep in mind that the first 4 bytes are consumed from DataInput (the
length) and `KeyValue.create` calls:
`in.readFully(bytes); return new KeyValue(bytes, 0, length);` - thus `bytes`
starts with four '0' bytes coming from `Import.write line 139:
out.writeInt(0);`!
The [KeyValue
constructure](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L343)
in turn calls
[`KeyValueUtil.checkKeyValueBytes`](https://github.com/apache/hbase/blob/6cfff27465620112eb308cf881e6321d928743ed/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValueUtil.java#L520).
And here I discovered that the serialization can never match the
deserialization because `KeyValueUtil.checkKeyValueBytes` expects the key len
at the beginning of the `bytes`buffer (note that `offset==0`) but this '0' as
mentioned before.
To sum it up:
`CellWritableComparable.write`writes:
[ [4 bytes key length], [4 bytes '0'], [...] ]
`CellWritableComparable.readFields` expects:
[ [4bytes key-value length], [4 bytes key length], [...] ]
Honestly, I wonder how that ever worked ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (HBASE-25837) TestRollingRestart is flaky
[ https://issues.apache.org/jira/browse/HBASE-25837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yulin Niu resolved HBASE-25837. --- Fix Version/s: 2.5.0 3.0.0-alpha-1 Resolution: Fixed > TestRollingRestart is flaky > --- > > Key: HBASE-25837 > URL: https://issues.apache.org/jira/browse/HBASE-25837 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Yulin Niu >Assignee: Yulin Niu >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > Attachments: screenshot-1.png > > > the method *waitForRSShutdownToStartAndFinish* is incorrect, for the judgment > method *sm.areDeadServersInProgress* may execute before scp is fired > !screenshot-1.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25837) TestRollingRestart is flaky
[ https://issues.apache.org/jira/browse/HBASE-25837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340778#comment-17340778 ] Yulin Niu commented on HBASE-25837: --- push to master and branch-2. Thanks [~zhangduo]'s reviewing > TestRollingRestart is flaky > --- > > Key: HBASE-25837 > URL: https://issues.apache.org/jira/browse/HBASE-25837 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Yulin Niu >Assignee: Yulin Niu >Priority: Major > Attachments: screenshot-1.png > > > the method *waitForRSShutdownToStartAndFinish* is incorrect, for the judgment > method *sm.areDeadServersInProgress* may execute before scp is fired > !screenshot-1.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25850) Fix spotbugs warnings on branch-2
[
https://issues.apache.org/jira/browse/HBASE-25850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang resolved HBASE-25850.
---
Fix Version/s: 2.5.0
Hadoop Flags: Reviewed
Resolution: Fixed
Merged to branch-2.
Thank [~xiaozhang] for contributing.
> Fix spotbugs warnings on branch-2
> -
>
> Key: HBASE-25850
> URL: https://issues.apache.org/jira/browse/HBASE-25850
> Project: HBase
> Issue Type: Bug
> Components: Compaction, findbugs, mob
>Reporter: Duo Zhang
>Assignee: Xiao Zhang
>Priority: Major
> Labels: beginner, trivial
> Fix For: 2.5.0
>
>
> {noformat}
> Dodgy code Warnings
> Code Warning
> UCUseless condition: it's known that major == true at this point
> Bug type UC_USELESS_CONDITION (click for details)
> In class org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor
> In method
> org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.performCompaction(Compactor$FileDetails,
> InternalScanner, CellSink, long, boolean, ThroughputController, boolean, int)
> Value major == true
> Unreachable code at DefaultMobStoreCompactor.java:[line 210]
> At DefaultMobStoreCompactor.java:[line 210]
> {noformat}
> https://github.com/apache/hbase/blob/afa9836b87f9c909ac4fc18d904a5b5b0a3dfff6/hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java#L211
> {code}
> if (major) {
> try {
> delFileWriter = mobStore.createDelFileWriterInTmp(new
> Date(fd.latestPutTs),
> fd.maxKeyCount, major ? majorCompactionCompression :
> minorCompactionCompression,
> store.getRegionInfo().getStartKey());
> } catch (IOException e) {
> LOG.warn(
> "Failed to create del writer, "
> + "we will continue the compaction by writing delete markers
> directly in store files",
> e);
> }
> }
> {code}
> The 'major ? majorCompactionCompression : minorCompactionCompression' is
> useless as we have a 'if (major)' test above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #3239: HBASE-25850 Fix spotbugs warnings on branch-2
Apache9 merged pull request #3239: URL: https://github.com/apache/hbase/pull/3239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 merged pull request #3220: HBASE-25837 TestRollingRestart is flaky
nyl3532016 merged pull request #3220: URL: https://github.com/apache/hbase/pull/3220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3220: HBASE-25837 TestRollingRestart is flaky
Apache-HBase commented on pull request #3220: URL: https://github.com/apache/hbase/pull/3220#issuecomment-834259453 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 7s | master passed | | +1 :green_heart: | compile | 1m 7s | master passed | | +1 :green_heart: | shadedjars | 9m 8s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | the patch passed | | +1 :green_heart: | compile | 1m 3s | the patch passed | | +1 :green_heart: | javac | 1m 3s | the patch passed | | +1 :green_heart: | shadedjars | 9m 12s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 220m 50s | hbase-server in the patch passed. | | | | 254m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3220 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 97e87c700b5a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6309c090b5 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/testReport/ | | Max. process+thread count | 3414 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3220: HBASE-25837 TestRollingRestart is flaky
Apache9 commented on pull request #3220: URL: https://github.com/apache/hbase/pull/3220#issuecomment-834235722 Let's try it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on a change in pull request #3232: HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map
wchevreuil commented on a change in pull request #3232:
URL: https://github.com/apache/hbase/pull/3232#discussion_r628068697
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
##
@@ -135,9 +135,7 @@ public CellWritableComparable(Cell kv) {
@Override
public void write(DataOutput out) throws IOException {
- out.writeInt(PrivateCellUtil.estimatedSerializedSizeOfKey(kv));
- out.writeInt(0);
- PrivateCellUtil.writeFlatKey(kv, out);
+KeyValueUtil.write(new KeyValue(kv), out);
Review comment:
You are basically reverting changes from commit
0a24178d06ac3dd5780522d52b5054e7aff1e50f from 4 years ago (HBASE-18649). Can
you get rid of that exception if you just remove the `out.writeInt(0)` there?
I'm not sure, though, on other potential side effects. Maybe we would also need
to change `Import.readFields` to stay aligned with HBASE-18649 (I guess we
could use `CellBuilder` to construct a `Cell` directly).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3220: HBASE-25837 TestRollingRestart is flaky
Apache-HBase commented on pull request #3220: URL: https://github.com/apache/hbase/pull/3220#issuecomment-834205035 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 9s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 8m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | +1 :green_heart: | shadedjars | 8m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 139m 18s | hbase-server in the patch passed. | | | | 170m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3220 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 85eb3ce0c732 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6309c090b5 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/testReport/ | | Max. process+thread count | 3882 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3239: HBASE-25850 Fix spotbugs warnings on branch-2
Apache-HBase commented on pull request #3239: URL: https://github.com/apache/hbase/pull/3239#issuecomment-834192519 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 56s | branch-2 passed | | +1 :green_heart: | compile | 1m 14s | branch-2 passed | | +1 :green_heart: | shadedjars | 7m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 35s | the patch passed | | +1 :green_heart: | compile | 1m 12s | the patch passed | | +1 :green_heart: | javac | 1m 12s | the patch passed | | +1 :green_heart: | shadedjars | 7m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 205m 12s | hbase-server in the patch passed. | | | | 236m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3239 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux be7e7d89c11a 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 2938ba421b | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/testReport/ | | Max. process+thread count | 2627 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25858) [branch-1] make hbase-thrift optional in hbase-assembly module
[
https://issues.apache.org/jira/browse/HBASE-25858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17340670#comment-17340670
]
Reid Chan commented on HBASE-25858:
---
QA complaint a lot, looks like we need to merge it to see exact effect, WDYT.
> [branch-1] make hbase-thrift optional in hbase-assembly module
> --
>
> Key: HBASE-25858
> URL: https://issues.apache.org/jira/browse/HBASE-25858
> Project: HBase
> Issue Type: Task
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Blocker
>
> {code}
> [ERROR] Failed to execute goal on project hbase-assembly: Could not resolve
> dependencies for project org.apache.hbase:hbase-assembly:pom:1.7.0: Failure
> to find org.apache.hbase:hbase-thrift:jar:1.7.0 in
> https://repository.apache.org/content/repositories/releases/ was cached in
> the local repository, resolution will not be reattempted until the update
> interval of apache release has elapsed or updates are forced -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #3239: HBASE-25850 Fix spotbugs warnings on branch-2
Apache-HBase commented on pull request #3239: URL: https://github.com/apache/hbase/pull/3239#issuecomment-834149557 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 44s | branch-2 passed | | +1 :green_heart: | compile | 0m 59s | branch-2 passed | | +1 :green_heart: | shadedjars | 6m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | shadedjars | 5m 56s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 144m 3s | hbase-server in the patch passed. | | | | 169m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3239 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fdb6f7006423 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 2938ba421b | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/testReport/ | | Max. process+thread count | 3940 (vs. ulimit of 12500) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3239/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3220: HBASE-25837 TestRollingRestart is flaky
Apache-HBase commented on pull request #3220: URL: https://github.com/apache/hbase/pull/3220#issuecomment-834131492 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 3m 24s | master passed | | +1 :green_heart: | checkstyle | 1m 10s | master passed | | +1 :green_heart: | spotbugs | 2m 15s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | the patch passed | | +1 :green_heart: | compile | 3m 31s | the patch passed | | +1 :green_heart: | javac | 3m 31s | the patch passed | | +1 :green_heart: | checkstyle | 1m 3s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 19m 11s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 2m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 15s | The patch does not generate ASF License warnings. | | | | 51m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3220 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 46d5403fa4ea 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 6309c090b5 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 96 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3220/4/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
