Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2014410707 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 6m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 54s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 41s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | shadedjars | 6m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 229m 52s | hbase-server in the patch passed. | | | | 261m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 366e61c19fae 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/testReport/ | | Max. process+thread count | 5484 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path [hbase]
Apache-HBase commented on PR #5771: URL: https://github.com/apache/hbase/pull/5771#issuecomment-2014409997 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 54s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | -1 :x: | mvninstall | 4m 9s | root in master failed. | | +1 :green_heart: | compile | 0m 47s | master passed | | +1 :green_heart: | shadedjars | 6m 26s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 32s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | shadedjars | 6m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 336m 14s | hbase-server in the patch failed. | | | | 365m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5771 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 25464d449771 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Temurin-1.8.0_352-b08 | | mvninstall | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk8-hadoop3-check/output/branch-mvninstall-root.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/testReport/ | | Max. process+thread count | 4435 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2014396728 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 59s | master passed | | +1 :green_heart: | compile | 1m 7s | master passed | | +1 :green_heart: | shadedjars | 5m 46s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 56s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 5m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 33s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 219m 2s | hbase-server in the patch passed. | | | | 246m 38s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c5b61ef4c091 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/testReport/ | | Max. process+thread count | 5173 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path [hbase]
Apache-HBase commented on PR #5771: URL: https://github.com/apache/hbase/pull/5771#issuecomment-2014396280 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 49s | master passed | | +1 :green_heart: | compile | 1m 5s | master passed | | +1 :green_heart: | shadedjars | 6m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 0m 58s | the patch passed | | +1 :green_heart: | javac | 0m 58s | the patch passed | | +1 :green_heart: | shadedjars | 6m 51s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 40s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 318m 42s | hbase-server in the patch failed. | | | | 350m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5771 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 330bea6e4136 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/testReport/ | | Max. process+thread count | 4453 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2014395329 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 0s | master passed | | +1 :green_heart: | compile | 1m 14s | master passed | | +1 :green_heart: | shadedjars | 5m 13s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 55s | the patch passed | | +1 :green_heart: | compile | 1m 14s | the patch passed | | +1 :green_heart: | javac | 1m 14s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 25s | hbase-client in the patch passed. | | -1 :x: | unit | 217m 24s | hbase-server in the patch failed. | | | | 244m 57s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 92ab850cfe00 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-17.0.10+7 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/testReport/ | | Max. process+thread count | 6082 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28314) Enable maven-source-plugin for all modules
[ https://issues.apache.org/jira/browse/HBASE-28314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated HBASE-28314: Fix Version/s: 4.0.0-alpha-1 > Enable maven-source-plugin for all modules > -- > > Key: HBASE-28314 > URL: https://issues.apache.org/jira/browse/HBASE-28314 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.6.0, 2.4.17, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 3.0.0-beta-2, 2.5.9 > > > There is no source jar generated for hbase-server. > Enabling maven-source-plugin seems to work fine, and Eclipse can use the > generated sources jar for debugging etc. > It seems that these source JARs are not generated at least since HBase 2.0.0, > but we did have source JARs sometime in the 1.x time frame. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihir Monani updated HBASE-26192:
-
Attachment: HBCK Report in JSON Format.png
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Mihir Monani
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
> Attachments: HBCK Report in JSON Format.png, Screen Shot 2022-05-31
> at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihir Monani updated HBASE-26192:
-
Attachment: (was: Sample output.png)
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Mihir Monani
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
> Attachments: HBCK Report in JSON Format.png, Screen Shot 2022-05-31
> at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihir Monani updated HBASE-26192:
-
Attachment: Sample output.png
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Mihir Monani
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
> Attachments: Sample output.png, Screen Shot 2022-05-31 at 5.18.15
> PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28314) Enable maven-source-plugin for all modules
[ https://issues.apache.org/jira/browse/HBASE-28314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated HBASE-28314: Description: There is no source jar generated for hbase-server. Enabling maven-source-plugin seems to work fine, and Eclipse can use the generated sources jar for debugging etc. It seems that these source JARs are not generated at least since HBase 2.0.0, but we did have source JARs sometime in the 1.x time frame. was: There is no source jar generated for hbase-server. Enabling maven-source-plugin seems to work fine, and Eclipse can't use the generated sources jar for debugging etc. It seems that these source JARs are not generated at least since HBase 2.0.0, but we did have source JARs sometime in the 1.x time frame. > Enable maven-source-plugin for all modules > -- > > Key: HBASE-28314 > URL: https://issues.apache.org/jira/browse/HBASE-28314 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.6.0, 2.4.17, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9 > > > There is no source jar generated for hbase-server. > Enabling maven-source-plugin seems to work fine, and Eclipse can use the > generated sources jar for debugging etc. > It seems that these source JARs are not generated at least since HBase 2.0.0, > but we did have source JARs sometime in the 1.x time frame. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28314) Enable maven-source-plugin for all modules
[ https://issues.apache.org/jira/browse/HBASE-28314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated HBASE-28314: Description: There is no source jar generated for hbase-server. Enabling maven-source-plugin seems to work fine, and Eclipse can't use the generated sources jar for debugging etc. It seems that these source JARs are not generated at least since HBase 2.0.0, but we did have source JARs sometime in the 1.x time frame. was: There is no source jar generated for hbase-server. Enabling maven-source-plugin seems to work fine, and Eclipse can use the generated sources jar for debugging etc. It seems that these source JARs are not generated at least since HBase 2.0.0, but we did have source JARs sometime in the 1.x time frame. > Enable maven-source-plugin for all modules > -- > > Key: HBASE-28314 > URL: https://issues.apache.org/jira/browse/HBASE-28314 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.6.0, 2.4.17, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9 > > > There is no source jar generated for hbase-server. > Enabling maven-source-plugin seems to work fine, and Eclipse can't use the > generated sources jar for debugging etc. > It seems that these source JARs are not generated at least since HBase 2.0.0, > but we did have source JARs sometime in the 1.x time frame. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-26192 Master UI hbck should provide a JSON formatted output option [hbase]
mihir6692 commented on PR #5772:
URL: https://github.com/apache/hbase/pull/5772#issuecomment-2014346223
Here is Sample Output for `/hbck/hbck_metrics` call
```
{
"data": {
"hbck_report_start_time": 1711082593359,
"hbck_report_end_time": 1711082593924,
"hbck_orphan_regions_on_fs": [
{
"orphan_region_id": "abcdefghi",
"orphan_region_hdfs_path": "hdfs://path1"
},
{
"orphan_region_id": "pqrsxyz",
"orphan_region_hdfs_path": "hdfs://path2"
}
],
"hbck_orphan_regions_on_rs": [
{
"orphan_region_id": "abcdefghi",
"orphan_region_rs_name": {
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
}
},
{
"orphan_region_id": "pqrsxyz",
"orphan_region_rs_name": {
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
}
}
],
"hbck_inconsistent_regions": [
{
"inconsistent_region_id": "abcdefef",
"server_name_in_meta": {
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
},
"list_of_server_name": [
{
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
},
{
"host_name": "localhost2",
"host_port": 16010,
"start_code": 12345
}
]
},
{
"inconsistent_region_id": "xyzpqrs",
"server_name_in_meta": {
"host_name": "localhost2",
"host_port": 16010,
"start_code": 12345
},
"list_of_server_name": [
{
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
},
{
"host_name": "localhost2",
"host_port": 16010,
"start_code": 12345
}
]
}
],
"hbck_holes": [
{
"region_hole1": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
},
"region_hole2": {
"region_id": "region2",
"table_name": "table2",
"start_key": "abcd",
"end_key": "xyz"
}
},
{
"region_hole1": {
"region_id": "region2",
"table_name": "table2",
"start_key": "abcd",
"end_key": "xyz"
},
"region_hole2": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
}
}
],
"hbck_overlaps": [
{
"overlap_region1": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
},
"overlap_region2": {
"region_id": "region2",
"table_name": "table2",
"start_key": "abcd",
"end_key": "xyz"
}
},
{
"overlap_region1": {
"region_id": "region2",
"table_name": "table2",
"start_key": "abcd",
"end_key": "xyz"
},
"overlap_region2": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
}
}
],
"hbck_unknown_servers": [
{
"unknown_server_region_id": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
},
"unknown_server_name": {
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
}
},
{
"unknown_server_region_id": {
"region_id": "region1",
"table_name": "table1",
"start_key": "abcd",
"end_key": "xyz"
},
"unknown_server_name": {
"host_name": "localhost1",
"host_port": 16010,
"start_code": 12345
}
}
],
"hbck_empty_region_info": [
{
"empty_region_info": "emtpyRegion1"
},
{
"empty_region_info": "emtpyRegion2"
}
]
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
Re: [PR] HBASE-26192 Master UI hbck should provide a JSON formatted output option [hbase]
Apache-HBase commented on PR #5772: URL: https://github.com/apache/hbase/pull/5772#issuecomment-2014338929 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 12s | master passed | | +1 :green_heart: | compile | 2m 31s | master passed | | +1 :green_heart: | checkstyle | 0m 35s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | compile | 2m 32s | the patch passed | | +1 :green_heart: | javac | 2m 32s | the patch passed | | -0 :warning: | checkstyle | 0m 34s | hbase-server: The patch generated 1 new + 7 unchanged - 0 fixed = 8 total (was 7) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 5m 11s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 29m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5772/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5772 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 9e2bc177f9fb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5772/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5772/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28314) Enable maven-source-plugin for all modules
[ https://issues.apache.org/jira/browse/HBASE-28314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28314: -- Fix Version/s: 2.6.0 2.4.18 3.0.0-beta-2 2.5.9 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to all active branches. Thanks [~stoty]! > Enable maven-source-plugin for all modules > -- > > Key: HBASE-28314 > URL: https://issues.apache.org/jira/browse/HBASE-28314 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 2.6.0, 2.4.17, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9 > > > There is no source jar generated for hbase-server. > Enabling maven-source-plugin seems to work fine, and Eclipse can use the > generated sources jar for debugging etc. > It seems that these source JARs are not generated at least since HBase 2.0.0, > but we did have source JARs sometime in the 1.x time frame. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28314 Enable maven-source-plugin for all modules [hbase]
Apache9 merged PR #5754: URL: https://github.com/apache/hbase/pull/5754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28314 Enable maven-source-plugin for all modules [hbase]
Apache9 merged PR #5753: URL: https://github.com/apache/hbase/pull/5753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28314 Enable maven-source-plugin for all modules [hbase]
Apache9 merged PR #5752: URL: https://github.com/apache/hbase/pull/5752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26192 Master UI hbck should provide a JSON formatted output option [hbase]
mihir6692 commented on PR #5772: URL: https://github.com/apache/hbase/pull/5772#issuecomment-2014309924 Hi @virajjasani @apurtell @ndimiduk Can I get review on this JIRA? (I have tried to include all the comments mentioned the PR https://github.com/apache/hbase/pull/4470 ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28416) Remove hbase-examples from hbase-assembly
[ https://issues.apache.org/jira/browse/HBASE-28416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-28416: -- Fix Version/s: 2.6.0 3.0.0-beta-2 (was: 4.0.0-alpha-1) Hadoop Flags: Reviewed Release Note: Remove hbase-examples from the release binaries. We will still publish it into maven central. Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to branch-2.6+. Thanks [~stoty]! [~bbeaudreault] I set the fix versions to 2.6.0, you can include this commit and try publish again. > Remove hbase-examples from hbase-assembly > - > > Key: HBASE-28416 > URL: https://issues.apache.org/jira/browse/HBASE-28416 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-beta-1 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 3.0.0-beta-2 > > > hbase-assembly is supposed to contain programming examples for HBase. > However, it is added to the assembly API, and becomes part of the HBase > distrbutions. > On hand this adds some potentially useful components and coprocessors to > HBase, on the other hand many of those are not production quality, and were > never meant to be used as-is. > It also adds the Curator libraries to the general HBase classpath, which are > used but by but a single example. > Removing hbase-examples from the assembly would fix both problems. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28314 Enable maven-source-plugin for all modules [hbase]
Apache9 merged PR #5751: URL: https://github.com/apache/hbase/pull/5751 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28314 Enable maven-source-plugin for all modules [hbase]
Apache9 merged PR #5748: URL: https://github.com/apache/hbase/pull/5748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26192) Master UI hbck should provide a JSON formatted output option
[
https://issues.apache.org/jira/browse/HBASE-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-26192:
---
Labels: pull-request-available (was: )
> Master UI hbck should provide a JSON formatted output option
>
>
> Key: HBASE-26192
> URL: https://issues.apache.org/jira/browse/HBASE-26192
> Project: HBase
> Issue Type: New Feature
>Reporter: Andrew Kyle Purtell
>Assignee: Mihir Monani
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.0.0-beta-2
>
> Attachments: Screen Shot 2022-05-31 at 5.18.15 PM.png
>
>
> It used to be possible to get hbck's verdict of cluster status from the
> command line, especially useful for headless deployments, i.e. without
> requiring a browser with sufficient connectivity to load a UI, or scrape
> information out of raw HTML, or write regex to comb over log4j output. The
> hbck tool's output wasn't particularly convenient to parse but it was
> straightforward to extract the desired information with a handful of regular
> expressions.
> HBCK2 has a different design philosophy than the old hbck, which is to serve
> as a collection of small and discrete recovery and repair functions, rather
> than attempt to be a universal repair tool. This makes a lot of sense and
> isn't the issue at hand. Unfortunately the old hbck's utility for reporting
> the current cluster health assessment has not been replaced either in whole
> or in part. Instead:
> {quote}
> HBCK2 is for fixes. For listings of inconsistencies or blockages in the
> running cluster, you go elsewhere, to the logs and UI of the running cluster
> Master. Once an issue has been identified, you use the HBCK2 tool to ask the
> Master to effect fixes or to skip-over bad state. Asking the Master to make
> the fixes rather than try and effect the repair locally in a fix-it tool's
> context is another important difference between HBCK2 and hbck1.
> {quote}
> Developing custom tooling to mine logs and scrape UI simply to gain a top
> level assessment of system health is unsatisfying. There should be a
> convenient means for querying the system if issues that rise to the level of
> _inconsistency_, in the hbck parlance, are believed to be present. It would
> be relatively simple to bring back the experience of invoking a command line
> tool to deliver a verdict. This could be added to the hbck2 tool itself but
> given that hbase-operator-tools is a separate project an intrinsic solution
> is desirable.
> An option that immediately comes to mind is modification of the Master's
> hbck.jsp page to provide a JSON formatted output option if the HTTP Accept
> header asks for text/json. However, looking at the source of hbck.jsp, it
> makes more sense to leave it as is and implement a convenient machine
> parseable output format elsewhere. This can be trivially accomplished with a
> new servlet. Like hbck.jsp the servlet implementation would get a reference
> to HbckChore and present the information this class makes available via its
> various getters.
> The machine parseable output is sufficient to enable headless hbck status
> checking but it still would be nice if we could provide operators a command
> line tool that formats the information for convenient viewing in a terminal.
> That part could be implemented in the hbck2 tool after this proposal is
> implemented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path [hbase]
Apache-HBase commented on PR #5771: URL: https://github.com/apache/hbase/pull/5771#issuecomment-2014304083 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 9s | master passed | | +1 :green_heart: | compile | 0m 54s | master passed | | +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 28s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 214m 46s | hbase-server in the patch passed. | | | | 238m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5771 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b63a189fa3f9 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-17.0.10+7 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/testReport/ | | Max. process+thread count | 5271 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2014191755 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 52s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 19s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 51s | master passed | | +1 :green_heart: | compile | 3m 49s | master passed | | +1 :green_heart: | checkstyle | 0m 56s | master passed | | +1 :green_heart: | spotless | 0m 52s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 44s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 56s | the patch passed | | +1 :green_heart: | compile | 3m 42s | the patch passed | | +1 :green_heart: | javac | 3m 42s | the patch passed | | +1 :green_heart: | checkstyle | 0m 55s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 6m 23s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 1m 12s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 23s | The patch does not generate ASF License warnings. | | | | 43m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux a84f39db9232 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28416 Remove hbase-examples from hbase-assembly (#5739) [hbase]
Apache9 merged PR #5755: URL: https://github.com/apache/hbase/pull/5755 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27826) Region split and merge time while offline is O(n) with respect to number of store files
[
https://issues.apache.org/jira/browse/HBASE-27826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829713#comment-17829713
]
Duo Zhang commented on HBASE-27826:
---
I think the discussion around the design doc is enough for us to take some
actions now.
1. File a new issue, to add a version field in the tracker file defination, so
when we find out that we are reading a tracker file with a higher version, we
will fail so end users will know that they should do something before
downgrading.
2. Links and back references should also be encoded into the tracker file
directly, but this is not fully covered by the design doc. We can discuss this
further and open new issues for this topic, as it does not affect split/merge.
[~wchevreuil]
3. For split/merge, most things are covered in the design doc and we have
discussed a lot, we can wait for the POC/draft PR from [~prathyu6] and move
forward.
Thoughts?
Thanks.
> Region split and merge time while offline is O(n) with respect to number of
> store files
> ---
>
> Key: HBASE-27826
> URL: https://issues.apache.org/jira/browse/HBASE-27826
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.4
>Reporter: Andrew Kyle Purtell
>Priority: Major
>
> This is a significant availability issue when HFiles are on S3. =
> HBASE-26079 ({_}Use StoreFileTracker when splitting and merging{_}) changed
> the split and merge table procedure implementations to indirect through the
> StoreFileTracker implementation when selecting HFiles to be merged or split,
> rather than directly listing those using file system APIs. It also changed
> the commit logic in HRegionFileSystem to add the link/ref files on resulting
> split or merged regions to the StoreFileTracker. However, the creation of a
> link file is still a filesystem operation and creating a “file” on S3 can
> take well over a second. If, for example there are 20 store files in a
> region, which is not uncommon, after the region is taken offline for a split
> (or merge) it may require more than 20 seconds to create the link files
> before the results can be brought back online, creating a severe availability
> problem. Splits and merges are supposed to be fast, completing in less than a
> second, certainly less than a few seconds. This has been true when HFiles are
> stored on HDFS only because file creation operations there are nearly
> instantaneous.
> There are two issues but both can be handled with modifications to the store
> file tracker interface and the file based store file tracker implementation.
> When the file based store file file tracker is enabled the HFile links should
> be virtual entities that only exist in the file manifest. We do not require
> physical files in the filesystem to serve as links now. That is the magic of
> the this file tracker, the manifest file replaces requirements to list the
> filesystem.
> Then, when splitting or merging, the HFile links should be collected into a
> list and committed in one batch using a new FILE file tracker interface,
> requiring only one update of the manifest file in S3, bringing the time
> requirement for this operation to O(1) down from O[n].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path [hbase]
Apache-HBase commented on PR #5771: URL: https://github.com/apache/hbase/pull/5771#issuecomment-2014095126 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 12s | master passed | | +1 :green_heart: | compile | 2m 35s | master passed | | +1 :green_heart: | checkstyle | 0m 37s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 32s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | the patch passed | | +1 :green_heart: | compile | 2m 32s | the patch passed | | +1 :green_heart: | javac | 2m 32s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 5m 13s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 29m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5771 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 77deb27c20d4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5771/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28448) CompressionTest hangs when run over a Ozone ofs path
[
https://issues.apache.org/jira/browse/HBASE-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HBASE-28448:
---
Labels: ozone pull-request-available (was: ozone)
> CompressionTest hangs when run over a Ozone ofs path
>
>
> Key: HBASE-28448
> URL: https://issues.apache.org/jira/browse/HBASE-28448
> Project: HBase
> Issue Type: Bug
>Reporter: Pratyush Bhatt
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: ozone, pull-request-available
> Attachments: hbase_ozone_compression.jstack
>
>
> If we run the Compression test over HDFS path, it works fine:
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> hdfs://ns1/tmp/dir1/dir2/test_file.txt snappy
> 24/03/20 06:08:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:08:43 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:44 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS {code}
> The command exits, but when the same is tried over a ofs path, the command
> hangs.
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> ofs://ozone1710862004/test-222compression-vol/compression-buck2/test_file.txt
> snappy
> 24/03/20 06:05:19 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/03/20 06:05:20 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/03/20 06:05:20 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:05:20 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Volume:
> test-222compression-vol, with om as owner and space quota set to -1 bytes,
> counts quota set to -1
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Bucket:
> test-222compression-vol/compression-buck2, with bucket layout
> FILE_SYSTEM_OPTIMIZED, om as owner, Versioning false, Storage Type set to
> DISK and Encryption set to false, Replication Type set to server-side default
> replication type, Namespace Quota set to -1, Space Quota set to -1
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 WARN impl.MetricsSystemImpl: HBase metrics system already
> initialized!
> 24/03/20 06:05:21 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
> 24/03/20 06:05:22 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS
> .
> .
> .{code}
> The command doesnt exit.
> Attaching the jstack of the process below:
> [^hbase_ozone_compression.jstack]
> cc: [~weichiu]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path [hbase]
jojochuang opened a new pull request, #5771: URL: https://github.com/apache/hbase/pull/5771 CompressionTest should close the FileSystem object otherwise some resources may not be released, which caused an Ozone output stream user thread to hang. Deployed on a cluster, ran the same command `hbase org.apache.hadoop.hbase.util.CompressionTest ofs://ozone/vol1/bucket1/test_file.txt snappy` to verify it no longer hangs on an Ozone path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28448) CompressionTest hangs when run over a Ozone ofs path
[
https://issues.apache.org/jira/browse/HBASE-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829708#comment-17829708
]
Wei-Chiu Chuang commented on HBASE-28448:
-
Another bug is in side HBase CompressionTest: the test should close FileSystem
object properly.
> CompressionTest hangs when run over a Ozone ofs path
>
>
> Key: HBASE-28448
> URL: https://issues.apache.org/jira/browse/HBASE-28448
> Project: HBase
> Issue Type: Bug
>Reporter: Pratyush Bhatt
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: ozone
> Attachments: hbase_ozone_compression.jstack
>
>
> If we run the Compression test over HDFS path, it works fine:
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> hdfs://ns1/tmp/dir1/dir2/test_file.txt snappy
> 24/03/20 06:08:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:08:43 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:44 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS {code}
> The command exits, but when the same is tried over a ofs path, the command
> hangs.
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> ofs://ozone1710862004/test-222compression-vol/compression-buck2/test_file.txt
> snappy
> 24/03/20 06:05:19 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/03/20 06:05:20 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/03/20 06:05:20 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:05:20 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Volume:
> test-222compression-vol, with om as owner and space quota set to -1 bytes,
> counts quota set to -1
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Bucket:
> test-222compression-vol/compression-buck2, with bucket layout
> FILE_SYSTEM_OPTIMIZED, om as owner, Versioning false, Storage Type set to
> DISK and Encryption set to false, Replication Type set to server-side default
> replication type, Namespace Quota set to -1, Space Quota set to -1
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 WARN impl.MetricsSystemImpl: HBase metrics system already
> initialized!
> 24/03/20 06:05:21 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
> 24/03/20 06:05:22 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS
> .
> .
> .{code}
> The command doesnt exit.
> Attaching the jstack of the process below:
> [^hbase_ozone_compression.jstack]
> cc: [~weichiu]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28448) CompressionTest hangs when run over a Ozone ofs path
[
https://issues.apache.org/jira/browse/HBASE-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang reassigned HBASE-28448:
---
Assignee: Wei-Chiu Chuang
> CompressionTest hangs when run over a Ozone ofs path
>
>
> Key: HBASE-28448
> URL: https://issues.apache.org/jira/browse/HBASE-28448
> Project: HBase
> Issue Type: Bug
>Reporter: Pratyush Bhatt
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: ozone
> Attachments: hbase_ozone_compression.jstack
>
>
> If we run the Compression test over HDFS path, it works fine:
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> hdfs://ns1/tmp/dir1/dir2/test_file.txt snappy
> 24/03/20 06:08:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:08:43 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:44 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS {code}
> The command exits, but when the same is tried over a ofs path, the command
> hangs.
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> ofs://ozone1710862004/test-222compression-vol/compression-buck2/test_file.txt
> snappy
> 24/03/20 06:05:19 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/03/20 06:05:20 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/03/20 06:05:20 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:05:20 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Volume:
> test-222compression-vol, with om as owner and space quota set to -1 bytes,
> counts quota set to -1
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Bucket:
> test-222compression-vol/compression-buck2, with bucket layout
> FILE_SYSTEM_OPTIMIZED, om as owner, Versioning false, Storage Type set to
> DISK and Encryption set to false, Replication Type set to server-side default
> replication type, Namespace Quota set to -1, Space Quota set to -1
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 WARN impl.MetricsSystemImpl: HBase metrics system already
> initialized!
> 24/03/20 06:05:21 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
> 24/03/20 06:05:22 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS
> .
> .
> .{code}
> The command doesnt exit.
> Attaching the jstack of the process below:
> [^hbase_ozone_compression.jstack]
> cc: [~weichiu]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HBASE-28454) Make Outputstream writeExecutor daemon threads
Wei-Chiu Chuang created HBASE-28454: --- Summary: Make Outputstream writeExecutor daemon threads Key: HBASE-28454 URL: https://issues.apache.org/jira/browse/HBASE-28454 Project: HBase Issue Type: Bug Reporter: Wei-Chiu Chuang Found via the HBase CompressionTest HBASE-28448. We should consider making the threads daemon, not user thread. Otherwise process may hang. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28448) CompressionTest hangs when run over a Ozone ofs path
[
https://issues.apache.org/jira/browse/HBASE-28448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829707#comment-17829707
]
Wei-Chiu Chuang commented on HBASE-28448:
-
Ok I found the problem. It's a bug in HBase CompressionTest but also a bug in
Ozone. Will open PRs accordingly.
> CompressionTest hangs when run over a Ozone ofs path
>
>
> Key: HBASE-28448
> URL: https://issues.apache.org/jira/browse/HBASE-28448
> Project: HBase
> Issue Type: Bug
>Reporter: Pratyush Bhatt
>Priority: Major
> Labels: ozone
> Attachments: hbase_ozone_compression.jstack
>
>
> If we run the Compression test over HDFS path, it works fine:
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> hdfs://ns1/tmp/dir1/dir2/test_file.txt snappy
> 24/03/20 06:08:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:08:43 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:08:43 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:43 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:08:44 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS {code}
> The command exits, but when the same is tried over a ofs path, the command
> hangs.
> {code:java}
> hbase org.apache.hadoop.hbase.util.CompressionTest
> ofs://ozone1710862004/test-222compression-vol/compression-buck2/test_file.txt
> snappy
> 24/03/20 06:05:19 INFO protocolPB.OmTransportFactory: Loading OM transport
> implementation
> org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified
> by configuration.
> 24/03/20 06:05:20 INFO client.ClientTrustManager: Loading certificates for
> client.
> 24/03/20 06:05:20 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/03/20 06:05:20 INFO impl.MetricsSystemImpl: HBase metrics system started
> 24/03/20 06:05:20 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Volume:
> test-222compression-vol, with om as owner and space quota set to -1 bytes,
> counts quota set to -1
> 24/03/20 06:05:20 INFO rpc.RpcClient: Creating Bucket:
> test-222compression-vol/compression-buck2, with bucket layout
> FILE_SYSTEM_OPTIMIZED, om as owner, Versioning false, Storage Type set to
> DISK and Encryption set to false, Replication Type set to server-side default
> replication type, Namespace Quota set to -1, Space Quota set to -1
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 INFO compress.CodecPool: Got brand-new compressor [.snappy]
> 24/03/20 06:05:21 WARN impl.MetricsSystemImpl: HBase metrics system already
> initialized!
> 24/03/20 06:05:21 INFO metrics.MetricRegistries: Loaded MetricRegistries
> class org.apache.ratis.metrics.dropwizard3.Dm3MetricRegistriesImpl
> 24/03/20 06:05:22 INFO compress.CodecPool: Got brand-new decompressor
> [.snappy]
> SUCCESS
> .
> .
> .{code}
> The command doesnt exit.
> Attaching the jstack of the process below:
> [^hbase_ozone_compression.jstack]
> cc: [~weichiu]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28453) Support a middle ground between the Average and Fixed interval rate limiters
[ https://issues.apache.org/jira/browse/HBASE-28453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Mattingly updated HBASE-28453: -- Description: h3. Background HBase quotas support two rate limiters: a "fixed" and an "average" interval rate limiter. h4. FixedIntervalRateLimiter The fixed interval rate limiter is simpler: it has a TimeUnit, say 1 second, and it refills a resource allotment on the recurring interval. So you may get 10 resources every second, and if you exhaust all 10 resources in the first millisecond of an interval then you will need to wait 999ms to acquire even 1 more resource. h4. AverageIntervalRateLimiter The average interval rate limiter, HBase's default, allows for more flexibly timed refilling of the resource allotment. Extending our previous example, say you have a 10 reads/sec quota and you have exhausted all 10 resources within 1ms of the last full refill. If you request 1 more read then, rather than returning a 999ms wait interval indicating the next full refill time, the rate limiter will recognize that you only need to wait 99ms before 1 read can be available. After 100ms has passed in aggregate since the last full refill, it will support the refilling of 1/10th the limit to facilitate the request for 1/10th the resources. h3. The Problems with Current RateLimiters The problem with the fixed interval rate limiter is that it is too strict from a latency perspective. It results in quota limits to which we cannot fully subscribe with any consistency. The problem with the average interval rate limiter is that, in practice, it is far too optimistic. For example, a real rate limiter might limit to 100MB/sec of read IO per machine. Any multigets that come in will require only a tiny fraction of this limit; for example, a 64kb block is only 0.06% of the total. As a result, the vast majority of wait intervals end up being tiny — like <5ms. This can actually cause an inverse of your intention, where setting up a throttle causes a DDOS of your RPC layer via continuous throttling and ~immediate retrying. I've discussed this problem in https://issues.apache.org/jira/browse/HBASE-28429 and proposed a minimum wait interval as the solution there; after some more thinking, I believe this new rate limiter would be a less hacky solution to this deficit so I'd like to close that Jira in favor of this one. See the attached chart where I put in place a 10k req/sec/machine throttle for this user at 10:43 to try to curb this high traffic, and it resulted in a huge spike of req/sec due to the throttle/retry loop created by the AverageIntervalRateLimiter. h3. Original Proposal: PartialIntervalRateLimiter as a Solution I've implemented a RateLimiter which allows for partial chunks of the overall interval to be refilled, by default these chunks are 10% (or 100ms of a 1s interval). I've deployed this to a test cluster at my day job and have seen this really help our ability to full subscribe to a quota limit without executing superfluous retries. See the other attached chart which shows a cluster undergoing a rolling restart from using FixedIntervalRateLimiter to my new PartialIntervalRateLimiter and how it is then able to fully subscribe to its allotted 25MB/sec/machine read IO quota. h3. Updated Proposal: Improving FixedIntervalRateLimiter Rather than implement a new rate limiter, we can make a lower touch change which just adds support for a refill interval that is less than the time unit on a FixedIntervalRateLimiter. This can be a no-op change for those who have not opted into the feature by having the refill interval default to the time unit. For clarity, see [my branch here|https://github.com/apache/hbase/compare/master...HubSpot:hbase:HBASE-28453] which I will PR soon was: h3. Background HBase quotas support two rate limiters: a "fixed" and an "average" interval rate limiter. h4. FixedIntervalRateLimiter The fixed interval rate limiter is simpler: it has a TimeUnit, say 1 second, and it refills a resource allotment on the recurring interval. So you may get 10 resources every second, and if you exhaust all 10 resources in the first millisecond of an interval then you will need to wait 999ms to acquire even 1 more resource. h4. AverageIntervalRateLimiter The average interval rate limiter, HBase's default, allows for more flexibly timed refilling of the resource allotment. Extending our previous example, say you have a 10 reads/sec quota and you have exhausted all 10 resources within 1ms of the last full refill. If you request 1 more read then, rather than returning a 999ms wait interval indicating the next full refill time, the rate limiter will recognize that you only need to wait 99ms before 1 read can be available. After 100ms has passed in aggregate since the last full refill, it will support the refilling of 1/10th the limit to facilitate the request for
[jira] [Commented] (HBASE-28444) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[
https://issues.apache.org/jira/browse/HBASE-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829686#comment-17829686
]
Hudson commented on HBASE-28444:
Results for branch master
[build #1036 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1036/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1036/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1036/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1036/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
> ---
>
> Key: HBASE-28444
> URL: https://issues.apache.org/jira/browse/HBASE-28444
> Project: HBase
> Issue Type: Task
> Components: security, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2013607363 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | shadedjars | 5m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 26s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | shadedjars | 5m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 23s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 227m 30s | hbase-server in the patch passed. | | | | 253m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux cc684c4e9ac7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/testReport/ | | Max. process+thread count | 5540 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2013576815 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 9s | master passed | | +1 :green_heart: | compile | 1m 6s | master passed | | +1 :green_heart: | shadedjars | 5m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 3s | the patch passed | | +1 :green_heart: | compile | 1m 6s | the patch passed | | +1 :green_heart: | javac | 1m 6s | the patch passed | | +1 :green_heart: | shadedjars | 5m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 35s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 219m 34s | hbase-server in the patch passed. | | | | 248m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 163f30226f1d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/testReport/ | | Max. process+thread count | 4529 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28450 BuckeCache.evictBlocksByHfileName won't work after a cache recovery from file [hbase]
Apache-HBase commented on PR #5769: URL: https://github.com/apache/hbase/pull/5769#issuecomment-2013336875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 33s | master passed | | +1 :green_heart: | compile | 1m 8s | master passed | | +1 :green_heart: | shadedjars | 6m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 29s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | shadedjars | 6m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 264m 14s | hbase-server in the patch failed. | | | | 292m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5769 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7f6dd825ec16 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/testReport/ | | Max. process+thread count | 4701 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28444) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[
https://issues.apache.org/jira/browse/HBASE-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829670#comment-17829670
]
Hudson commented on HBASE-28444:
Results for branch branch-2.6
[build #79 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/79/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/79/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/79/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/79/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.6/79/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
> ---
>
> Key: HBASE-28444
> URL: https://issues.apache.org/jira/browse/HBASE-28444
> Project: HBase
> Issue Type: Task
> Components: security, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28450 BuckeCache.evictBlocksByHfileName won't work after a cache recovery from file [hbase]
Apache-HBase commented on PR #5769: URL: https://github.com/apache/hbase/pull/5769#issuecomment-2013309223 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 2s | master passed | | +1 :green_heart: | compile | 0m 51s | master passed | | +1 :green_heart: | shadedjars | 6m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 30s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 44s | the patch passed | | +1 :green_heart: | compile | 0m 52s | the patch passed | | +1 :green_heart: | javac | 0m 52s | the patch passed | | +1 :green_heart: | shadedjars | 6m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 259m 7s | hbase-server in the patch failed. | | | | 284m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5769 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b7ce4876b60d 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/testReport/ | | Max. process+thread count | 4863 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (HBASE-28453) Support a middle ground between the Average and Fixed interval rate limiters
[ https://issues.apache.org/jira/browse/HBASE-28453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Mattingly reassigned HBASE-28453: - Assignee: Ray Mattingly > Support a middle ground between the Average and Fixed interval rate limiters > > > Key: HBASE-28453 > URL: https://issues.apache.org/jira/browse/HBASE-28453 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.6.0 >Reporter: Ray Mattingly >Assignee: Ray Mattingly >Priority: Major > Attachments: Screenshot 2024-03-21 at 2.08.51 PM.png, Screenshot > 2024-03-21 at 2.30.01 PM.png > > > h3. Background > HBase quotas support two rate limiters: a "fixed" and an "average" interval > rate limiter. > h4. FixedIntervalRateLimiter > The fixed interval rate limiter is simpler: it has a TimeUnit, say 1 second, > and it refills a resource allotment on the recurring interval. So you may get > 10 resources every second, and if you exhaust all 10 resources in the first > millisecond of an interval then you will need to wait 999ms to acquire even 1 > more resource. > h4. AverageIntervalRateLimiter > The average interval rate limiter, HBase's default, allows for more flexibly > timed refilling of the resource allotment. Extending our previous example, > say you have a 10 reads/sec quota and you have exhausted all 10 resources > within 1ms of the last full refill. If you request 1 more read then, rather > than returning a 999ms wait interval indicating the next full refill time, > the rate limiter will recognize that you only need to wait 99ms before 1 read > can be available. After 100ms has passed in aggregate since the last full > refill, it will support the refilling of 1/10th the limit to facilitate the > request for 1/10th the resources. > h3. The Problems with Current RateLimiters > The problem with the fixed interval rate limiter is that it is too strict > from a latency perspective. It results in quota limits to which we cannot > fully subscribe with any consistency. > The problem with the average interval rate limiter is that, in practice, it > is far too optimistic. For example, a real rate limiter might limit to > 100MB/sec of read IO per machine. Any multigets that come in will require > only a tiny fraction of this limit; for example, a 64kb block is only 0.06% > of the total. As a result, the vast majority of wait intervals end up being > tiny — like <5ms. This can actually cause an inverse of your intention, where > setting up a throttle causes a DDOS of your RPC layer via continuous > throttling and ~immediate retrying. I've discussed this problem in > https://issues.apache.org/jira/browse/HBASE-28429 and proposed a minimum wait > interval as the solution there; after some more thinking, I believe this new > rate limiter would be a less hacky solution to this deficit so I'd like to > close that Jira in favor of this one. > See the attached chart where I put in place a 10k req/sec/machine throttle > for this user at 10:43 to try to curb this high traffic, and it resulted in a > huge spike of req/sec due to the throttle/retry loop created by the > AverageIntervalRateLimiter. > h3. PartialIntervalRateLimiter as a Solution > I've implemented a RateLimiter which allows for partial chunks of the overall > interval to be refilled, by default these chunks are 10% (or 100ms of a 1s > interval). I've deployed this to a test cluster at my day job and have seen > this really help our ability to full subscribe to a quota limit without > executing superfluous retries. See the other attached chart which shows a > cluster undergoing a rolling restart from using FixedIntervalRateLimiter to > my new PartialIntervalRateLimiter and how it is then able to fully subscribe > to its allotted 25MB/sec/machine read IO quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28429) Quotas should have a configurable minimum wait interval
[ https://issues.apache.org/jira/browse/HBASE-28429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829667#comment-17829667 ] Ray Mattingly commented on HBASE-28429: --- I think we should close this and instead favor https://issues.apache.org/jira/browse/HBASE-28453 > Quotas should have a configurable minimum wait interval > --- > > Key: HBASE-28429 > URL: https://issues.apache.org/jira/browse/HBASE-28429 > Project: HBase > Issue Type: Improvement >Reporter: Ray Mattingly >Assignee: Ray Mattingly >Priority: Major > > At my day job we're attempting to rollout read size throttling by default for > thousands of distinct users across hundreds of multi-tenant clusters. > During our rollout we've observed that throttles with a 1 second refill > interval will yield relatively tiny wait intervals disproportionately often. > From what we've seen, wait intervals are <=5ms on approximately 20-50% of our > RpcThrottlingExceptions; this could sound theoretically promising if latency > is your top priority. But, in reality, this makes it very difficult to > configure a throttle tolerant HBase client because retries become very prone > to near-immediate exhaustion, and throttled clients quickly saturate the > cluster's RPC layer with rapid-fire retries. > One can combat this with the FixedIntervalRateLimiter, but that's a very > heavy handed approach from latency's perspective, and can still yield tiny > intervals that exhaust retries and erroneously fail client operations under > significant load. > With this in mind, I'm proposing that we introduce a configurable minimum > wait interval for quotas, defaulted to 0. This would make quotas much more > usable at scale from our perspective. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28453) Support a middle ground between the Average and Fixed interval rate limiters
Ray Mattingly created HBASE-28453: - Summary: Support a middle ground between the Average and Fixed interval rate limiters Key: HBASE-28453 URL: https://issues.apache.org/jira/browse/HBASE-28453 Project: HBase Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Ray Mattingly Attachments: Screenshot 2024-03-21 at 2.08.51 PM.png, Screenshot 2024-03-21 at 2.30.01 PM.png h3. Background HBase quotas support two rate limiters: a "fixed" and an "average" interval rate limiter. h4. FixedIntervalRateLimiter The fixed interval rate limiter is simpler: it has a TimeUnit, say 1 second, and it refills a resource allotment on the recurring interval. So you may get 10 resources every second, and if you exhaust all 10 resources in the first millisecond of an interval then you will need to wait 999ms to acquire even 1 more resource. h4. AverageIntervalRateLimiter The average interval rate limiter, HBase's default, allows for more flexibly timed refilling of the resource allotment. Extending our previous example, say you have a 10 reads/sec quota and you have exhausted all 10 resources within 1ms of the last full refill. If you request 1 more read then, rather than returning a 999ms wait interval indicating the next full refill time, the rate limiter will recognize that you only need to wait 99ms before 1 read can be available. After 100ms has passed in aggregate since the last full refill, it will support the refilling of 1/10th the limit to facilitate the request for 1/10th the resources. h3. The Problems with Current RateLimiters The problem with the fixed interval rate limiter is that it is too strict from a latency perspective. It results in quota limits to which we cannot fully subscribe with any consistency. The problem with the average interval rate limiter is that, in practice, it is far too optimistic. For example, a real rate limiter might limit to 100MB/sec of read IO per machine. Any multigets that come in will require only a tiny fraction of this limit; for example, a 64kb block is only 0.06% of the total. As a result, the vast majority of wait intervals end up being tiny — like <5ms. This can actually cause an inverse of your intention, where setting up a throttle causes a DDOS of your RPC layer via continuous throttling and ~immediate retrying. I've discussed this problem in https://issues.apache.org/jira/browse/HBASE-28429 and proposed a minimum wait interval as the solution there; after some more thinking, I believe this new rate limiter would be a less hacky solution to this deficit so I'd like to close that Jira in favor of this one. See the attached chart where I put in place a 10k req/sec/machine throttle for this user at 10:43 to try to curb this high traffic, and it resulted in a huge spike of req/sec due to the throttle/retry loop created by the AverageIntervalRateLimiter. h3. PartialIntervalRateLimiter as a Solution I've implemented a RateLimiter which allows for partial chunks of the overall interval to be refilled, by default these chunks are 10% (or 100ms of a 1s interval). I've deployed this to a test cluster at my day job and have seen this really help our ability to full subscribe to a quota limit without executing superfluous retries. See the other attached chart which shows a cluster undergoing a rolling restart from using FixedIntervalRateLimiter to my new PartialIntervalRateLimiter and how it is then able to fully subscribe to its allotted 25MB/sec/machine read IO quota. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28450 BuckeCache.evictBlocksByHfileName won't work after a cache recovery from file [hbase]
Apache-HBase commented on PR #5769: URL: https://github.com/apache/hbase/pull/5769#issuecomment-2013177863 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 8s | master passed | | +1 :green_heart: | compile | 0m 51s | master passed | | +1 :green_heart: | shadedjars | 5m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | the patch passed | | +1 :green_heart: | compile | 0m 52s | the patch passed | | +1 :green_heart: | javac | 0m 52s | the patch passed | | +1 :green_heart: | shadedjars | 5m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 204m 55s | hbase-server in the patch failed. | | | | 228m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5769 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f3b4af633531 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-17.0.10+7 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/testReport/ | | Max. process+thread count | 4820 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28444) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[
https://issues.apache.org/jira/browse/HBASE-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829647#comment-17829647
]
Hudson commented on HBASE-28444:
Results for branch branch-2.5
[build #500 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/500/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/500/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/500/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/500/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/500/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
> ---
>
> Key: HBASE-28444
> URL: https://issues.apache.org/jira/browse/HBASE-28444
> Project: HBase
> Issue Type: Task
> Components: security, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28444) Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
[
https://issues.apache.org/jira/browse/HBASE-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829642#comment-17829642
]
Hudson commented on HBASE-28444:
Results for branch branch-2.4
[build #710 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/710/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/710/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/710/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/710/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/710/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4
> ---
>
> Key: HBASE-28444
> URL: https://issues.apache.org/jira/browse/HBASE-28444
> Project: HBase
> Issue Type: Task
> Components: security, Zookeeper
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-2, 2.5.9
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2012883032 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 18s | master passed | | +1 :green_heart: | compile | 3m 13s | master passed | | +1 :green_heart: | checkstyle | 0m 51s | master passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 13s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 55s | the patch passed | | +1 :green_heart: | compile | 3m 12s | the patch passed | | +1 :green_heart: | javac | 3m 12s | the patch passed | | +1 :green_heart: | checkstyle | 0m 50s | the patch passed | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 5m 29s | Patch does not cause any errors with Hadoop 3.3.6. | | -1 :x: | spotless | 0m 36s | patch has 22 errors when running spotless:check, run spotless:apply to fix. | | +1 :green_heart: | spotbugs | 2m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 34m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 8f25af25b6a8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | whitespace | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/artifact/yetus-general-check/output/whitespace-eol.txt | | spotless | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/artifact/yetus-general-check/output/patch-spotless.txt | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2012656493 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 39s | master passed | | +1 :green_heart: | compile | 0m 57s | master passed | | +1 :green_heart: | shadedjars | 5m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 26s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed | | +1 :green_heart: | javac | 0m 56s | the patch passed | | -1 :x: | shadedjars | 2m 22s | patch has 10 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 24s | hbase-client in the patch passed. | | -1 :x: | unit | 227m 39s | hbase-server in the patch failed. | | | | 250m 56s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 19d98a4f92cc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Temurin-1.8.0_352-b08 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/testReport/ | | Max. process+thread count | 5452 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28450 BuckeCache.evictBlocksByHfileName won't work after a cache recovery from file [hbase]
Apache-HBase commented on PR #5769: URL: https://github.com/apache/hbase/pull/5769#issuecomment-2012475719 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 38s | master passed | | +1 :green_heart: | compile | 2m 52s | master passed | | +1 :green_heart: | checkstyle | 0m 38s | master passed | | +1 :green_heart: | spotless | 0m 53s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 48s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 29s | the patch passed | | +1 :green_heart: | compile | 2m 57s | the patch passed | | +1 :green_heart: | javac | 2m 57s | the patch passed | | +1 :green_heart: | checkstyle | 0m 42s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 5m 41s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 0m 48s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 15s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 33m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5769 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux fbfd8b2eaca2 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5769/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache-HBase commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2012088346 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 0s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 36s | master passed | | +1 :green_heart: | compile | 3m 45s | master passed | | +1 :green_heart: | checkstyle | 0m 58s | master passed | | +1 :green_heart: | spotless | 0m 47s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | +1 :green_heart: | compile | 3m 48s | the patch passed | | +1 :green_heart: | javac | 3m 48s | the patch passed | | +1 :green_heart: | checkstyle | 1m 0s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 6m 19s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 0m 52s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 7s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 20s | The patch generated 1 ASF License warnings. | | | | 40m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5770 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 9abb9a6a5d72 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / ade6ab2148 | | Default Java | Eclipse Adoptium-11.0.17+8 | | asflicense | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 77 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5770/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28443 Return too slow when scanning a table with non-existing REGION_REPLICA_ID [hbase]
Apache9 commented on PR #5767: URL: https://github.com/apache/hbase/pull/5767#issuecomment-2011978374 Mind explaining more on why locating is so slow when the replica id does not exist? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28444 Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 [hbase]
dependabot[bot] commented on PR #5763: URL: https://github.com/apache/hbase/pull/5763#issuecomment-2011967855 OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. If you change your mind, just re-open this PR and I'll resolve any conflicts on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28444 Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 [hbase]
Apache9 closed pull request #5763: HBASE-28444 Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 URL: https://github.com/apache/hbase/pull/5763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28444 Bump org.apache.zookeeper:zookeeper from 3.8.3 to 3.8.4 [hbase]
Apache9 commented on PR #5763: URL: https://github.com/apache/hbase/pull/5763#issuecomment-2011967803 Close as duplicated by #5765 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache9 commented on PR #5770: URL: https://github.com/apache/hbase/pull/5770#issuecomment-2011965780 @stoty FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28436) Use connection url to specify the connection registry information
[ https://issues.apache.org/jira/browse/HBASE-28436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28436: --- Labels: pull-request-available (was: ) > Use connection url to specify the connection registry information > - > > Key: HBASE-28436 > URL: https://issues.apache.org/jira/browse/HBASE-28436 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > > As describe in this email from [~ndimiduk] > https://lists.apache.org/thread/98wqlkqvlnmpx3r7yrg9mw4pqz9ppofh > The first advantage here is that, we can encode the connection registry > implementation in the scheme of the connection url, so for replication, we > can now support cluster key other than zookeeper, which is important for us > to remove zookeeper dependency on our public facing APIs. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[PR] HBASE-28436 Use connection url to specify the connection registry inf… [hbase]
Apache9 opened a new pull request, #5770: URL: https://github.com/apache/hbase/pull/5770 …ormation -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-27763) Recover WAL encounter KeeperErrorCode = NoNode cause RegionServer crash
[ https://issues.apache.org/jira/browse/HBASE-27763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-27763: --- Labels: pull-request-available (was: ) > Recover WAL encounter KeeperErrorCode = NoNode cause RegionServer crash > > > Key: HBASE-27763 > URL: https://issues.apache.org/jira/browse/HBASE-27763 > Project: HBase > Issue Type: Bug >Reporter: guoxiaojiao >Priority: Major > Labels: pull-request-available > > 20696 2022-07-07 14:51:52,638 ERROR.HRegionServer: * ABORTING region > server ***: Failed to operate on replication queue * > 120697 org.apache.hadoop.hbase.replication.ReplicationException: Failed to > set log position (serverName=***, > queueId=***.regiongroup-0.1657176199216,position=55122787) > 120698 at > org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:261) > 120699 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.lambda$logPositionAndCleanOldLogs$7(ReplicationSourceManager.java:529) > 120700 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.interruptOrAbortWhenFail(ReplicationSourceManager.java:476) > 120701 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:528) > 120702 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:271) > 120703 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:205) > 120704 at > org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:120) > 120705 Caused by: org.apache.zookeeper.KeeperException$NoNodeException: > KeeperErrorCode = NoNode > 120706 at > org.apache.zookeeper.KeeperException.create(KeeperException.java:118) > 120707 at > org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:1925) > 120708 at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:1830) > 120709 at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.multi(RecoverableZooKeeper.java:668) > 120710 at > org.apache.hadoop.hbase.zookeeper.ZKUtil.multiOrSequential(ZKUtil.java:1797) > 120711 at > org.apache.hadoop.hbase.replication.ZKReplicationQueueStorage.setWALPosition(ZKReplicationQueueStorage.java:251) > 120712 ... 6 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27763 Recover WAL encounter KeeperErrorCode = NoNode cause Regi… [hbase]
thangTang commented on PR #5177: URL: https://github.com/apache/hbase/pull/5177#issuecomment-2011753444 Hi @gottagogottagoGxj, appreciate if you could give some more explain about this ticket and your HBase version. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
