[jira] [Comment Edited] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509613#comment-17509613
]
Lars Hofhansl edited comment on HBASE-26812 at 3/21/22, 5:52 AM:
-
Thanks [~comnetwork].
Getting the user showed up the profiler, but that might have just obscured the
actual problem you mentioned.
PHOENIX-6671 is a one-liner that I can easily undo locally and I'm happy to
test an HBase fix here. I'll try you patch tomorrow.
(And apologies that I don't have much time to fix the HBase problem myself. I
used to be much more active in the HBase community. :( )
was (Author: lhofhansl):
Thanks [~comnetwork].
Getting the user showed up the profiler, but that might have just obscured the
actual problem you mentioned.
PHOENIX-6671 is a one-liner that I can easily undo locally and I'm happy to
test an HBase fix here.
(And apologies that I don't have much time to fix the HBase problem myself. I
used to be much more active in the HBase community. :( )
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509613#comment-17509613
]
Lars Hofhansl commented on HBASE-26812:
---
Thanks [~comnetwork].
Getting the user showed up the profiler, but that might have just obscured the
actual problem you mentioned.
PHOENIX-6671 is a one-liner that I can easily undo locally and I'm happy to
test an HBase fix here.
(And apologies that I don't have much time to fix the HBase problem myself. I
used to be much more active in the HBase community. :( )
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4253: HBASE-26872 Load rate calculator for cost functions should be more pr…
Apache-HBase commented on pull request #4253: URL: https://github.com/apache/hbase/pull/4253#issuecomment-1073496596 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 36s | master passed | | +1 :green_heart: | compile | 0m 24s | master passed | | +1 :green_heart: | shadedjars | 5m 40s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 18s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 40s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | +1 :green_heart: | javac | 0m 23s | the patch passed | | +1 :green_heart: | shadedjars | 6m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 12m 26s | hbase-balancer in the patch passed. | | | | 35m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4253 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 1bc43c365842 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d7638d66d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/testReport/ | | Max. process+thread count | 161 (vs. ulimit of 3) | | modules | C: hbase-balancer U: hbase-balancer | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4253: HBASE-26872 Load rate calculator for cost functions should be more pr…
Apache-HBase commented on pull request #4253: URL: https://github.com/apache/hbase/pull/4253#issuecomment-1073495736 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 40s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 42s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 4m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 17s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | | +1 :green_heart: | compile | 0m 17s | the patch passed | | +1 :green_heart: | javac | 0m 17s | the patch passed | | +1 :green_heart: | shadedjars | 4m 41s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 14s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 10m 10s | hbase-balancer in the patch passed. | | | | 30m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4253 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux ffe4e57827db 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d7638d66d | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/testReport/ | | Max. process+thread count | 190 (vs. ulimit of 3) | | modules | C: hbase-balancer U: hbase-balancer | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4253: HBASE-26872 Load rate calculator for cost functions should be more pr…
Apache-HBase commented on pull request #4253: URL: https://github.com/apache/hbase/pull/4253#issuecomment-1073493906 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 19s | master passed | | +1 :green_heart: | compile | 0m 33s | master passed | | +1 :green_heart: | checkstyle | 0m 13s | master passed | | +1 :green_heart: | spotbugs | 0m 27s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | the patch passed | | +1 :green_heart: | compile | 0m 26s | the patch passed | | +1 :green_heart: | javac | 0m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 14s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 0m 35s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 30m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4253 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 23996c647168 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d7638d66d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-balancer U: hbase-balancer | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4253/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4252: HBASE-25768 Support an overall simple and fast balance strategy for St…
Apache-HBase commented on pull request #4252: URL: https://github.com/apache/hbase/pull/4252#issuecomment-1073481615 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 21s | master passed | | +1 :green_heart: | compile | 3m 47s | master passed | | +1 :green_heart: | checkstyle | 1m 10s | master passed | | +1 :green_heart: | spotbugs | 2m 37s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 16s | the patch passed | | +1 :green_heart: | compile | 3m 45s | the patch passed | | -0 :warning: | javac | 0m 31s | hbase-balancer generated 4 new + 17 unchanged - 0 fixed = 21 total (was 17) | | -0 :warning: | checkstyle | 0m 11s | hbase-balancer: The patch generated 2 new + 75 unchanged - 1 fixed = 77 total (was 76) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 15m 6s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 45m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4252/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4252 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 66e7175452ef 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d7638d66d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4252/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-balancer.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4252/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-balancer.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-balancer hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4252/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly opened a new pull request #4253: HBASE-26872 Load rate calculator for cost functions should be more pr…
sunhelly opened a new pull request #4253: URL: https://github.com/apache/hbase/pull/4253 …ecise -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25768) Support an overall simple and fast balance strategy for StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha updated HBASE-25768: --- Summary: Support an overall simple and fast balance strategy for StochasticLoadBalancer (was: Support a overall simple and fast balance strategy for StochasticLoadBalancer) > Support an overall simple and fast balance strategy for StochasticLoadBalancer > -- > > Key: HBASE-25768 > URL: https://issues.apache.org/jira/browse/HBASE-25768 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 3.0.0-alpha-1, 2.0.0, 1.4.13 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > When we use StochasticLoadBalancer + balanceByTable, we could face two > difficulties. > # For each table, their regions are distributed uniformly, but for the > overall cluster, still exiting imbalance between RSes; > # When there are large-scaled restart of RSes, or expansion for groups or > cluster, we hope the balancer can execute as soon as possible, but the > StochasticLoadBalancer may need a lot of time to compute costs. > We can detect these circumstances in StochasticLoadBalancer(such as using the > percentage of skew tables), and before the normal balance steps trying, we > can add a strategy to let it just balance like the SimpleLoadBalancer or use > few light cost functions here. > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-25768) Support an overall simple and fast balance strategy for StochasticLoadBalancer
[ https://issues.apache.org/jira/browse/HBASE-25768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha updated HBASE-25768: --- Status: Patch Available (was: Open) > Support an overall simple and fast balance strategy for StochasticLoadBalancer > -- > > Key: HBASE-25768 > URL: https://issues.apache.org/jira/browse/HBASE-25768 > Project: HBase > Issue Type: Improvement > Components: Balancer >Affects Versions: 1.4.13, 2.0.0, 3.0.0-alpha-1 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > When we use StochasticLoadBalancer + balanceByTable, we could face two > difficulties. > # For each table, their regions are distributed uniformly, but for the > overall cluster, still exiting imbalance between RSes; > # When there are large-scaled restart of RSes, or expansion for groups or > cluster, we hope the balancer can execute as soon as possible, but the > StochasticLoadBalancer may need a lot of time to compute costs. > We can detect these circumstances in StochasticLoadBalancer(such as using the > percentage of skew tables), and before the normal balance steps trying, we > can add a strategy to let it just balance like the SimpleLoadBalancer or use > few light cost functions here. > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] sunhelly opened a new pull request #4252: HBASE-25768 Support a overall simple and fast balance strategy for St…
sunhelly opened a new pull request #4252: URL: https://github.com/apache/hbase/pull/4252 …ochasticLoadBalancer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4249: HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo…
Apache-HBase commented on pull request #4249: URL: https://github.com/apache/hbase/pull/4249#issuecomment-1073456042 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 44s | master passed | | +1 :green_heart: | compile | 3m 31s | master passed | | +1 :green_heart: | checkstyle | 1m 0s | master passed | | +1 :green_heart: | spotbugs | 2m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 45s | the patch passed | | +1 :green_heart: | compile | 3m 34s | the patch passed | | +1 :green_heart: | javac | 3m 34s | the patch passed | | +1 :green_heart: | checkstyle | 1m 1s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 19s | The patch does not generate ASF License warnings. | | | | 42m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4249 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 53df18ad624d 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 0d7638d66d | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/2/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-26870) Log4j2 integration is incorrect in nighly's client integration test
[ https://issues.apache.org/jira/browse/HBASE-26870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26870. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.5+. Thanks [~busbey] for reviewing! > Log4j2 integration is incorrect in nighly's client integration test > --- > > Key: HBASE-26870 > URL: https://issues.apache.org/jira/browse/HBASE-26870 > Project: HBase > Issue Type: Bug > Components: jenkins, logging >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.5.0, 3.0.0-alpha-3 > > > On master it still uses log4j2.xml and on branch-2.x we completely misses > this part... > Let me prepare PRs to fix them. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26872) Load rate calculator for cost functions should be more precise
[
https://issues.apache.org/jira/browse/HBASE-26872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26872:
---
Description:
The rate calculator should avoid negative values, e.g. if a region is moved
from RS1 to RS2, the request count loads in the balancer cache maybe
[100,200,0,100,200], then the region load cost calculated by
CostFromRegionLoadAsRateFunction#getRegionLoadCost will be
(100-200+100+100)/4=25, while the real cost is (100+0+100+200)/4=100.
{code:java}
protected double getRegionLoadCost(Collection
regionLoadList) {
Iterator iter = regionLoadList.iterator();
if (!iter.hasNext()) {
return 0;
}
double previous = getCostFromRl(iter.next());
if (!iter.hasNext()) {
return 0;
}
double cost = 0;
do {
double current = getCostFromRl(iter.next());
cost += current - previous;
previous = current;
} while (iter.hasNext());
return Math.max(0, cost / (regionLoadList.size() - 1));
} {code}
We should change the cost accumulate codes to,
{code:java}
cost += current >= previous ? current - previous : current; {code}
was:
The rate calculator should avoid negative values, e.g. if a region is moved
from RS1 to RS2, the request count loads in the balancer cache maybe
[100,200,0,100,200], then the region load cost calculated by
CostFromRegionLoadAsRateFunction#getRegionLoadCost will be
(100-200+100+100)/4=25, while the real cost is (100+100+200)/4=100.
{code:java}
protected double getRegionLoadCost(Collection
regionLoadList) {
Iterator iter = regionLoadList.iterator();
if (!iter.hasNext()) {
return 0;
}
double previous = getCostFromRl(iter.next());
if (!iter.hasNext()) {
return 0;
}
double cost = 0;
do {
double current = getCostFromRl(iter.next());
cost += current - previous;
previous = current;
} while (iter.hasNext());
return Math.max(0, cost / (regionLoadList.size() - 1));
} {code}
We should change the cost accumulate codes to,
{code:java}
cost += current >= previous ? current - previous : current; {code}
> Load rate calculator for cost functions should be more precise
> --
>
> Key: HBASE-26872
> URL: https://issues.apache.org/jira/browse/HBASE-26872
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> The rate calculator should avoid negative values, e.g. if a region is moved
> from RS1 to RS2, the request count loads in the balancer cache maybe
> [100,200,0,100,200], then the region load cost calculated by
> CostFromRegionLoadAsRateFunction#getRegionLoadCost will be
> (100-200+100+100)/4=25, while the real cost is (100+0+100+200)/4=100.
> {code:java}
> protected double getRegionLoadCost(Collection
> regionLoadList) {
> Iterator iter = regionLoadList.iterator();
> if (!iter.hasNext()) {
> return 0;
> }
> double previous = getCostFromRl(iter.next());
> if (!iter.hasNext()) {
> return 0;
> }
> double cost = 0;
> do {
> double current = getCostFromRl(iter.next());
> cost += current - previous;
> previous = current;
> } while (iter.hasNext());
> return Math.max(0, cost / (regionLoadList.size() - 1));
> } {code}
> We should change the cost accumulate codes to,
> {code:java}
> cost += current >= previous ? current - previous : current; {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26868) Introduce read/write request size metrics
[ https://issues.apache.org/jira/browse/HBASE-26868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509559#comment-17509559 ] Xiaolin Ha commented on HBASE-26868: Thanks, [~bbeaudreault] . The original purpose I introduced read/write requests size to metrics is to design request size cost functions for the StochasticLoadBalancer. And I used the same counter as the request count in our company. If the worry here is the number overflow, we can reset the counter to 0 when it is negative, just like when the region moved or RS aborted the request count is reset. I think the key point here is that the accumulated counter result is not as important as the rate that the counter changes(for balancer and QPS). While the counter is used in the balancer cost functions, so the rate calculator should handle the decrease of the counters, e.g. [100,200,0,100,200], as describe in HBASE-26872. > Introduce read/write request size metrics > - > > Key: HBASE-26868 > URL: https://issues.apache.org/jira/browse/HBASE-26868 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.4.11 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > Calculating the read and write size(unit is bytes) in the region, and > aggregated in table and regionserver levels. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (HBASE-26872) Load rate calculator for cost functions should be more precise
Xiaolin Ha created HBASE-26872:
--
Summary: Load rate calculator for cost functions should be more
precise
Key: HBASE-26872
URL: https://issues.apache.org/jira/browse/HBASE-26872
Project: HBase
Issue Type: Improvement
Components: Balancer
Affects Versions: 2.4.11
Reporter: Xiaolin Ha
Assignee: Xiaolin Ha
The rate calculator should avoid negative values, e.g. if a region is moved
from RS1 to RS2, the request count loads in the balancer cache maybe
[100,200,0,100,200], then the region load cost calculated by
CostFromRegionLoadAsRateFunction#getRegionLoadCost will be
(100-200+100+100)/4=25, while the real cost is (100+100+200)/4=100.
{code:java}
protected double getRegionLoadCost(Collection
regionLoadList) {
Iterator iter = regionLoadList.iterator();
if (!iter.hasNext()) {
return 0;
}
double previous = getCostFromRl(iter.next());
if (!iter.hasNext()) {
return 0;
}
double cost = 0;
do {
double current = getCostFromRl(iter.next());
cost += current - previous;
previous = current;
} while (iter.hasNext());
return Math.max(0, cost / (regionLoadList.size() - 1));
} {code}
We should change the cost accumulate codes to,
{code:java}
cost += current >= previous ? current - previous : current; {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache9 merged pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 merged pull request #4250: URL: https://github.com/apache/hbase/pull/4250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 merged pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 merged pull request #4251: URL: https://github.com/apache/hbase/pull/4251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
Apache9 commented on a change in pull request #4125:
URL: https://github.com/apache/hbase/pull/4125#discussion_r830695471
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/NettyRpcConnection.java
##
@@ -297,11 +295,10 @@ public void operationComplete(ChannelFuture future)
throws Exception {
established(ch);
}
}
- }).channel();
+ }).sync().channel();
Review comment:
There is a BufferCallBeforeInitHandler to handle the case for writing
rpc call out before initializing the connection.
https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/BufferCallBeforeInitHandler.java
That's why we can avoid calling sync in this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
joshelser commented on a change in pull request #4125:
URL: https://github.com/apache/hbase/pull/4125#discussion_r830692050
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/io/crypto/tls/X509Util.java
##
@@ -0,0 +1,320 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.crypto.tls;
+
+import java.io.BufferedInputStream;
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.security.GeneralSecurityException;
+import java.security.KeyManagementException;
+import java.security.KeyStore;
+import java.security.NoSuchAlgorithmException;
+import java.security.Security;
+import java.security.cert.PKIXBuilderParameters;
+import java.security.cert.X509CertSelector;
+import java.util.Objects;
+import java.util.concurrent.atomic.AtomicReference;
+import javax.net.ssl.CertPathTrustManagerParameters;
+import javax.net.ssl.KeyManager;
+import javax.net.ssl.KeyManagerFactory;
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManager;
+import javax.net.ssl.TrustManagerFactory;
+import javax.net.ssl.X509ExtendedTrustManager;
+import javax.net.ssl.X509KeyManager;
+import javax.net.ssl.X509TrustManager;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.common.X509Exception;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Utility code for X509 handling
+ *
+ * Default cipher suites:
+ *
+ * Performance testing done by Facebook engineers shows that on Intel x86_64
machines,
+ * Java9 performs better with GCM and Java8 performs better with CBC, so
these seem like
+ * reasonable defaults.
+ */
+@InterfaceAudience.Private
+public class X509Util {
+
+ private static final Logger LOG =
+LoggerFactory.getLogger(X509Util.class);
+
+ // Config
+ static final String CONFIG_PREFIX = "hbase.security.tls.";
+ static final String TLS_CONFIG_PROTOCOL = CONFIG_PREFIX + "protocol";
+ static final String TLS_CONFIG_KEYSTORE_LOCATION = CONFIG_PREFIX +
"keystore.location";
+ static final String TLS_CONFIG_KEYSTORE_TYPE = CONFIG_PREFIX +
"keystore.type";
+ static final String TLS_CONFIG_KEYSTORE_PASSWORD = CONFIG_PREFIX +
"keystore.password";
+ static final String TLS_CONFIG_TRUSTSTORE_LOCATION = CONFIG_PREFIX +
"truststore.location";
+ static final String TLS_CONFIG_TRUSTSTORE_TYPE = CONFIG_PREFIX +
"truststore.type";
+ static final String TLS_CONFIG_TRUSTSTORE_PASSWORD = CONFIG_PREFIX +
"truststore.password";
+ static final String TLS_CONFIG_CLR = CONFIG_PREFIX + "clr";
+ static final String TLS_CONFIG_OCSP = CONFIG_PREFIX + "ocsp";
+
+ public static final String HBASE_NETTY_RPCSERVER_TLS_ENABLED =
+"hbase.netty.rpcserver.tls.enabled";
+
+ public static final String DEFAULT_PROTOCOL = "TLSv1.2";
+ private static String[] getGCMCiphers() {
+return new String[]{"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
+ "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
+ "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384"};
+ }
+
+ private static String[] getCBCCiphers() {
+return new String[]{"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
+ "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
+ "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
+ "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
+ "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA"};
+ }
+
+ private static String[] concatArrays(String[] left, String[] right) {
+String[] result = new String[left.length + right.length];
+System.arraycopy(left, 0, result, 0, left.length);
+System.arraycopy(right, 0, result, left.length, right.length);
+return result;
+ }
+
+ // On Java 8, prefer CBC ciphers since AES-NI support is lacking and GCM is
slower than CBC.
+ private static final String[] DEFAULT_CIPHERS_JAVA8 =
concatArrays(getCBCCiphers(),
+getGCMCiphers());
+ // On Java 9 and later, prefer GCM ciphers due to improved AES-NI support.
+ // Note that this performance assumption might not hold true
[GitHub] [hbase] joshelser commented on pull request #4125: HBASE-26666 Add native TLS encryption support to RPC server/client
joshelser commented on pull request #4125: URL: https://github.com/apache/hbase/pull/4125#issuecomment-1073372011 > does anybody know why HBase not using Epoll (native transport) for client sockets? I'm not sure actually. Maybe @Apache9 knows. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #4064: HBASE-26667 Integrate user-experience for hbase-client
joshelser commented on a change in pull request #4064:
URL: https://github.com/apache/hbase/pull/4064#discussion_r830680098
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/security/token/OAuthBearerTokenUtil.java
##
@@ -68,8 +74,44 @@ public static void addTokenForUser(User user, String
encodedToken, long lifetime
}
};
subject.getPrivateCredentials().add(jwt);
+if (LOG.isDebugEnabled()) {
+ LOG.debug("JWT token has been added to user credentials with expiry
{}",
+lifetimeMs == 0 ? "0" :
Instant.ofEpochMilli(lifetimeMs).toString());
+}
return null;
}
});
}
+
+ /**
+ * Check whether an OAuth Beaerer token is provided in environment variable
HADOOP_JWT.
+ * Parse and add it to user private credentials, but only if another token
is not already present.
+ */
+ public static void addTokenFromEnvironmentVar(User user, String token) {
+Optional> oauthBearerToken = user.getTokens().stream()
+ .filter((t) -> new Text(OAuthBearerUtils.TOKEN_KIND).equals(t.getKind()))
+ .findFirst();
+
+if (oauthBearerToken.isPresent()) {
+ return;
+}
+
+String[] tokens = token.split(",");
+if (StringUtils.isEmpty(tokens[0])) {
+ return;
+}
+long lifetimeMs = 0;
+if (tokens.length > 1) {
+ try {
+ZonedDateTime lifetime = ZonedDateTime.parse(tokens[1]);
+lifetimeMs = lifetime.toInstant().toEpochMilli();
+ } catch (DateTimeParseException e) {
+LOG.warn("Unable to parse JWT expiry: {}", tokens[1]);
Review comment:
> Theoretically it's also possible to parse the expiry field from the
JWT itself, but the question here is do you want to do that without signature
validation? If not, the client will also have to be capable of JWT validation
like the server.
Yeah, I would not suggest that we push the validation logic into the client
(at least without a very good reason)
> but lifetimeMs is a must for this to work.
If `lifetimeMs` is going to be important for the token selection to work, I
think we should just throw an Exception if we can't parse the timestamp.
Otherwise, we'll get questions later about why renewal/expiration/selection
logic didn't work correctly.
If we're going to focus on Knox as the authorization server, let's build the
implementation as to how Knox works. As we want to support other authz servers,
it will be easier to codify "supported features" of different authorization
servers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #4052: HBASE-25955 Setting NAMESPACES when adding a replication peer still requires scope definition at CF level
joshelser commented on a change in pull request #4052:
URL: https://github.com/apache/hbase/pull/4052#discussion_r830672891
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationPeerConfigUtil.java
##
@@ -290,12 +290,14 @@ public static ReplicationPeerConfig
convert(ReplicationProtos.ReplicationPeer pe
peer.getTableCfsList().toArray(new
ReplicationProtos.TableCF[peer.getTableCfsCount()]));
if (tableCFsMap != null) {
builder.setTableCFsMap(tableCFsMap);
+ builder.setChainedFiltersOperation(peer.getChainOperator());
Review comment:
Should apply validation here (in addition to or instead of Ruby) as the
Java API to set the chainOperator as a user could be writing Java code directly
instead of writing Ruby code to interact with HBase.
##
File path: hbase-shell/src/main/ruby/shell/commands/add_peer.rb
##
@@ -34,11 +34,14 @@ def help
to the peer cluster.
An optional parameter for table column families identifies which tables and/or
column families
will be replicated to the peer cluster.
+An optional parameter for the boolean operator to be applied over different
WAL Entry filters. If
+omitted, conjunction (AND) is applied.
Review comment:
I'm wondering if, through the shell, we should provide some more
simplicity for the operator. They are unaware of any of the WALFilters that we
are setting behind the scenes. To them, this operator would be nothing more
than a "magic word" (e.g. "I put 'OR' and then my data gets replicated"). I
guess it's better to get this code committed and then think about ways to make
it more clear to admins.
##
File path: hbase-shell/src/main/ruby/hbase/replication_admin.rb
##
@@ -67,6 +67,7 @@ def add_peer(id, args = {}, peer_tableCFs = nil)
peer_state = args.fetch(STATE, nil)
remote_wal_dir = args.fetch(REMOTE_WAL_DIR, nil)
serial = args.fetch(SERIAL, nil)
+chain_operator = args.fetch(OPERATOR, nil)
Review comment:
```suggestion
chain_operator = args.fetch(OPERATOR, nil).upcase
```
To normalize `chain_operator`. Ease of use if an operator types `and`
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/ChainWALEntryFilter.java
##
@@ -56,6 +59,13 @@ public ChainWALEntryFilter(List filters) {
initCellFilters();
}
+ public ChainWALEntryFilter(List filters, String
operatorName) {
+this(filters);
+if (!StringUtils.isEmpty(operatorName)) {
+ this.operator = Operator.valueOf(operatorName);
Review comment:
I was trying to figure out the first place we read the String
"operatorName" and make sure it fails gracefully.
I know you have the client-side checking in Ruby code, and I suggested we
have Java data validation. We should check it here as future-proofing. I think
this happens early enough in the replication setup that the client would see a
RemoteException flowing back to them? (not that their `add_peer` call would
succeed and just not replicate any data because it failed).
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/NamespaceTableCfWALEntryFilter.java
##
@@ -56,8 +56,9 @@ public Cell filterCell(final Entry entry, Cell cell) {
if (CellUtil.matchingColumn(cell, WALEdit.METAFAMILY, WALEdit.BULK_LOAD)) {
// If the cell is about BULKLOAD event, unpack and filter it by
BulkLoadCellFilter.
return bulkLoadFilter.filterCell(cell, fam ->
!peerConfig.needToReplicate(tableName, fam));
-} else {
+} else if ((!CellUtil.matchingFamily(cell, WALEdit.METAFAMILY))){
Review comment:
```suggestion
} else if (!CellUtil.matchingFamily(cell, WALEdit.METAFAMILY)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4243: HBASE-26812 ShortCircuitingClusterConnection fails to close RegionSca…
Apache-HBase commented on pull request #4243: URL: https://github.com/apache/hbase/pull/4243#issuecomment-1073346290 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 45s | branch-2 passed | | +1 :green_heart: | compile | 2m 20s | branch-2 passed | | +1 :green_heart: | shadedjars | 0m 47s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 38s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | shadedjars | 0m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 5m 6s | hbase-client in the patch passed. | | -1 :x: | unit | 416m 30s | hbase-server in the patch failed. | | | | 446m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4243 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4616ea20db9f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/testReport/ | | Max. process+thread count | 2127 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on pull request #3913: HBASE-26536 Checkstyle: permit single-line if-then statements
joshelser commented on pull request #3913: URL: https://github.com/apache/hbase/pull/3913#issuecomment-1073346088 Perhaps this is stalled by now, but I revived the email thread :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] joshelser commented on a change in pull request #2843: HBASE-25461 when the cluster has many tables, UI can be opened quickly
joshelser commented on a change in pull request #2843:
URL: https://github.com/apache/hbase/pull/2843#discussion_r830669632
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStates.java
##
@@ -804,4 +806,21 @@ public static String regionNamesToString(final
Collection regions) {
sb.append("]");
return sb.toString();
}
+
+ /**
+ * WEB UI
+ */
+ public synchronized Map getRegionLocations(TableName
tableName) {
+Map tableRegions = new TreeMap<>();
+for (RegionStateNode regionStateNode :
regionsMap.tailMap(tableName.getName()).values()) {
+ if (!regionStateNode.getTable().equals(tableName)) {
+break;
+ }
+ if (regionStateNode.toRegionState().isSplitOrMerge()) {
+continue;
+ }
Review comment:
We previously would see splitting/merged regions for a specific table,
right? The RegionState should be listed in the table of Regions for this HBase
table -- are not showing split/merge regions a big performance impact?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26666) Address bearer token being sent over wire before RPC encryption is enabled
[ https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509516#comment-17509516 ] Bryan Beaudreault commented on HBASE-2: --- By the way, I see the fix version here is [HBASE-26553|https://issues.apache.org/jira/issues/?jql=project+%3D+HBASE+AND+fixVersion+%3D+HBASE-26553]. I realize you're working on a larger feature here, but any chance this can go to branch-2 instead. It's more generally useful for those who want to work towards TLS/SSL vs SASL > Address bearer token being sent over wire before RPC encryption is enabled > -- > > Key: HBASE-2 > URL: https://issues.apache.org/jira/browse/HBASE-2 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Andor Molnar >Priority: Major > Fix For: HBASE-26553 > > > Today, HBase must complete the SASL handshake (saslClient.complete()) prior > to turning on any RPC encryption (hbase.rpc.protection=privacy, > sasl.QOP=auth-conf). > This is a problem because we have to transmit the bearer token to the server > before we can complete the sasl handshake. This would mean that we would > insecurely transmit the bearer token (which is equivalent to any other > password) which is a bad smell. > Ideally, if we can solve this problem for the oauth bearer mechanism, we > could also apply it to our delegation token interface for digest-md5 (which, > I believe, suffers the same problem). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26666) Address bearer token being sent over wire before RPC encryption is enabled
[ https://issues.apache.org/jira/browse/HBASE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509516#comment-17509516 ] Bryan Beaudreault edited comment on HBASE-2 at 3/20/22, 8:23 PM: - By the way, I see the fix version here is [HBASE-26553|https://issues.apache.org/jira/issues/?jql=project+%3D+HBASE+AND+fixVersion+%3D+HBASE-26553]. I realize you're working on a larger feature here, but any chance this can go to master/branch-2 instead. It's more generally useful for those who want to work towards TLS/SSL vs SASL was (Author: bbeaudreault): By the way, I see the fix version here is [HBASE-26553|https://issues.apache.org/jira/issues/?jql=project+%3D+HBASE+AND+fixVersion+%3D+HBASE-26553]. I realize you're working on a larger feature here, but any chance this can go to branch-2 instead. It's more generally useful for those who want to work towards TLS/SSL vs SASL > Address bearer token being sent over wire before RPC encryption is enabled > -- > > Key: HBASE-2 > URL: https://issues.apache.org/jira/browse/HBASE-2 > Project: HBase > Issue Type: Sub-task >Reporter: Josh Elser >Assignee: Andor Molnar >Priority: Major > Fix For: HBASE-26553 > > > Today, HBase must complete the SASL handshake (saslClient.complete()) prior > to turning on any RPC encryption (hbase.rpc.protection=privacy, > sasl.QOP=auth-conf). > This is a problem because we have to transmit the bearer token to the server > before we can complete the sasl handshake. This would mean that we would > insecurely transmit the bearer token (which is equivalent to any other > password) which is a bad smell. > Ideally, if we can solve this problem for the oauth bearer mechanism, we > could also apply it to our delegation token interface for digest-md5 (which, > I believe, suffers the same problem). -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26813) Remove javax.ws.rs-api dependency
[
https://issues.apache.org/jira/browse/HBASE-26813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509510#comment-17509510
]
Hudson commented on HBASE-26813:
Results for branch branch-2.5
[build #69 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/69//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Remove javax.ws.rs-api dependency
> -
>
> Key: HBASE-26813
> URL: https://issues.apache.org/jira/browse/HBASE-26813
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> I see that we still have {{javax.ws.rs-api}} as a dependency in our
> hbase-http pom. If, for example, the {{ClientBuilder}} from this jar is used,
> it'll instantiate whatever non-shaded jersey client is on the class path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] joshelser commented on pull request #4202: HBASE-26791 Memstore flush fencing issue for SFT
joshelser commented on pull request #4202: URL: https://github.com/apache/hbase/pull/4202#issuecomment-1073339512 > The start code of the 'new' RS may be less than the 'old' RS, as here, 'new' and 'old' just mean we reassigned a region from the 'old' to 'new', does not mean the 'new' RS is started after the 'old' RS. Oh that's tricky. I didn't even think about that case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26813) Remove javax.ws.rs-api dependency
[
https://issues.apache.org/jira/browse/HBASE-26813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509505#comment-17509505
]
Hudson commented on HBASE-26813:
Results for branch master
[build #542 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/542/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/542/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/542/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/542/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove javax.ws.rs-api dependency
> -
>
> Key: HBASE-26813
> URL: https://issues.apache.org/jira/browse/HBASE-26813
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> I see that we still have {{javax.ws.rs-api}} as a dependency in our
> hbase-http pom. If, for example, the {{ClientBuilder}} from this jar is used,
> it'll instantiate whatever non-shaded jersey client is on the class path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4243: HBASE-26812 ShortCircuitingClusterConnection fails to close RegionSca…
Apache-HBase commented on pull request #4243: URL: https://github.com/apache/hbase/pull/4243#issuecomment-1073290515 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 13s | branch-2 passed | | +1 :green_heart: | compile | 1m 8s | branch-2 passed | | +1 :green_heart: | shadedjars | 0m 22s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 46s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 36s | the patch passed | | +1 :green_heart: | compile | 1m 9s | the patch passed | | +1 :green_heart: | javac | 1m 9s | the patch passed | | +1 :green_heart: | shadedjars | 0m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 7s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 194m 15s | hbase-server in the patch passed. | | | | 211m 4s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4243 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f2fe0dc30c9a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/testReport/ | | Max. process+thread count | 3040 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4249: HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo…
Apache-HBase commented on pull request #4249: URL: https://github.com/apache/hbase/pull/4249#issuecomment-1073283433 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 18s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 21s | master passed | | +1 :green_heart: | compile | 0m 54s | master passed | | +1 :green_heart: | shadedjars | 3m 41s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 13s | the patch passed | | +1 :green_heart: | compile | 0m 54s | the patch passed | | +1 :green_heart: | javac | 0m 54s | the patch passed | | +1 :green_heart: | shadedjars | 3m 34s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 29s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 194m 56s | hbase-server in the patch passed. | | | | 213m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4249 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux a1b2d92506c3 5.4.0-96-generic #109-Ubuntu SMP Wed Jan 12 16:49:16 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/testReport/ | | Max. process+thread count | 2816 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4251: URL: https://github.com/apache/hbase/pull/4251#issuecomment-1073263524 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 46s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 5m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4251 | | Optional Tests | | | uname | Linux d58e021cb2a4 5.4.0-1068-aws #72~18.04.1-Ubuntu SMP Thu Mar 3 08:49:49 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Max. process+thread count | 36 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4251: URL: https://github.com/apache/hbase/pull/4251#issuecomment-1073262950 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 5s | Maven dependency ordering for patch | | -0 :warning: | shellcheck | 0m 1s | The patch generated 1 new + 1 unchanged - 1 fixed = 2 total (was 2) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 35s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4251 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 8f076feb327b 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | shellcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/artifact/yetus-general-check/output/diff-patch-shellcheck.txt | | Max. process+thread count | 32 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4251: URL: https://github.com/apache/hbase/pull/4251#issuecomment-1073262865 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 45s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4251 | | Optional Tests | | | uname | Linux 1d35cdcef779 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Max. process+thread count | 29 (vs. ulimit of 3) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4251/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4202: HBASE-26791 Memstore flush fencing issue for SFT
Apache9 commented on pull request #4202: URL: https://github.com/apache/hbase/pull/4202#issuecomment-1073260551 Any other concerns? @wchevreuil @joshelser -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4250: URL: https://github.com/apache/hbase/pull/4250#issuecomment-1073260448 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 7s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4250 | | Optional Tests | | | uname | Linux 2020fa29b1e8 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Max. process+thread count | 40 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4250: URL: https://github.com/apache/hbase/pull/4250#issuecomment-1073260394 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | shelldocs | 0m 0s | Shelldocs was not available. | | +1 :green_heart: | @author | 0m 1s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 6s | Maven dependency ordering for patch | | -0 :warning: | shellcheck | 0m 2s | The patch generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | +0 :ok: | asflicense | 0m 0s | ASF License check generated no output? | | | | 2m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4250 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux aeb8659ce0a8 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | shellcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/artifact/yetus-general-check/output/diff-patch-shellcheck.txt | | Max. process+thread count | 33 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/console | | versions | git=2.17.1 maven=3.6.3 shellcheck=0.4.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #4202: HBASE-26791 Memstore flush fencing issue for SFT
Apache9 commented on a change in pull request #4202:
URL: https://github.com/apache/hbase/pull/4202#discussion_r830622759
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/storefiletracker/StoreFileListFile.java
##
@@ -114,23 +127,96 @@ private int select(StoreFileList[] lists) {
return lists[0].getTimestamp() >= lists[1].getTimestamp() ? 0 : 1;
}
- StoreFileList load() throws IOException {
+ // file sequence id to path
+ private NavigableMap> listFiles() throws IOException {
+FileSystem fs = ctx.getRegionFileSystem().getFileSystem();
+FileStatus[] statuses;
+try {
+ statuses = fs.listStatus(trackFileDir);
+} catch (FileNotFoundException e) {
+ LOG.debug("Track file directory {} does not exist", trackFileDir, e);
+ return Collections.emptyNavigableMap();
+}
+if (statuses == null || statuses.length == 0) {
+ return Collections.emptyNavigableMap();
+}
+TreeMap> map = new TreeMap<>((l1, l2) ->
l2.compareTo(l1));
+for (FileStatus status : statuses) {
+ Path file = status.getPath();
+ if (!status.isFile()) {
+LOG.warn("Found invalid track file {}, which is not a file", file);
+continue;
+ }
+ if (!TRACK_FILE_PATTERN.matcher(file.getName()).matches()) {
+LOG.warn("Found invalid track file {}, skip", file);
+continue;
+ }
+ List parts =
Splitter.on(TRACK_FILE_SEPARATOR).splitToList(file.getName());
+ map.computeIfAbsent(Long.parseLong(parts.get(1)), k -> new
ArrayList<>()).add(file);
+}
+return map;
+ }
+
+ private void initializeTrackFiles(long seqId) {
+trackFiles[0] = new Path(trackFileDir, TRACK_FILE_PREFIX +
TRACK_FILE_SEPARATOR + seqId);
+trackFiles[1] = new Path(trackFileDir, TRACK_FILE_ROTATE_PREFIX +
TRACK_FILE_SEPARATOR + seqId);
+ }
+
+ private void cleanUpTrackFiles(long loadedSeqId,
+NavigableMap> seqId2TrackFiles) {
+FileSystem fs = ctx.getRegionFileSystem().getFileSystem();
+NavigableMap> toDelete =
+ loadedSeqId >= 0 ? seqId2TrackFiles.tailMap(loadedSeqId, false) :
seqId2TrackFiles;
+toDelete.values().stream().flatMap(l -> l.stream()).forEach(file -> {
+ ForkJoinPool.commonPool().execute(() -> {
+try {
+ fs.delete(file, false);
+} catch (IOException e) {
+ LOG.warn("failed to delete unused track file {}", file, e);
Review comment:
I've add a info log above which logs the loadedSeqId. Here the sequence
id for this file is contained in its name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache-HBase commented on pull request #4250: URL: https://github.com/apache/hbase/pull/4250#issuecomment-1073260375 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 26s | Maven dependency ordering for branch | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | ||| _ Other Tests _ | | | | 2m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4250 | | Optional Tests | | | uname | Linux 6d7246121990 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Max. process+thread count | 33 (vs. ulimit of 12500) | | modules | C: U: | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4250/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on a change in pull request #4202: HBASE-26791 Memstore flush fencing issue for SFT
Apache9 commented on a change in pull request #4202:
URL: https://github.com/apache/hbase/pull/4202#discussion_r830622653
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/storefiletracker/StoreFileListFile.java
##
@@ -114,23 +127,96 @@ private int select(StoreFileList[] lists) {
return lists[0].getTimestamp() >= lists[1].getTimestamp() ? 0 : 1;
}
- StoreFileList load() throws IOException {
+ // file sequence id to path
+ private NavigableMap> listFiles() throws IOException {
+FileSystem fs = ctx.getRegionFileSystem().getFileSystem();
+FileStatus[] statuses;
+try {
+ statuses = fs.listStatus(trackFileDir);
+} catch (FileNotFoundException e) {
+ LOG.debug("Track file directory {} does not exist", trackFileDir, e);
+ return Collections.emptyNavigableMap();
+}
+if (statuses == null || statuses.length == 0) {
+ return Collections.emptyNavigableMap();
+}
+TreeMap> map = new TreeMap<>((l1, l2) ->
l2.compareTo(l1));
+for (FileStatus status : statuses) {
+ Path file = status.getPath();
+ if (!status.isFile()) {
+LOG.warn("Found invalid track file {}, which is not a file", file);
+continue;
+ }
+ if (!TRACK_FILE_PATTERN.matcher(file.getName()).matches()) {
+LOG.warn("Found invalid track file {}, skip", file);
+continue;
+ }
+ List parts =
Splitter.on(TRACK_FILE_SEPARATOR).splitToList(file.getName());
+ map.computeIfAbsent(Long.parseLong(parts.get(1)), k -> new
ArrayList<>()).add(file);
+}
+return map;
+ }
+
+ private void initializeTrackFiles(long seqId) {
+trackFiles[0] = new Path(trackFileDir, TRACK_FILE_PREFIX +
TRACK_FILE_SEPARATOR + seqId);
Review comment:
Added a info log after creating them. The sequence id is contained in
the file name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 commented on pull request #4251: URL: https://github.com/apache/hbase/pull/4251#issuecomment-1073259959 See https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870/2/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #4251: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 opened a new pull request #4251: URL: https://github.com/apache/hbase/pull/4251 …ation test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 commented on pull request #4250: URL: https://github.com/apache/hbase/pull/4250#issuecomment-1073259856 See: https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870-branch-2/2/ Although the build is still failing, but at lease we could see the log output to find out why https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870-branch-2/2/artifact/output-integration/hadoop-3/mr-importtsv.err/*view*/ Filed HBASE-26871 for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 opened a new pull request #4250: HBASE-26870 Log4j2 integration is incorrect in nighly's client integr…
Apache9 opened a new pull request #4250: URL: https://github.com/apache/hbase/pull/4250 …ation test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
[
https://issues.apache.org/jira/browse/HBASE-26871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509464#comment-17509464
]
Duo Zhang commented on HBASE-26871:
---
[~apurtell] [~busbey] FYI.
> Client integration test fails on branch-2.x with ClassNotFoundException
> ---
>
> Key: HBASE-26871
> URL: https://issues.apache.org/jira/browse/HBASE-26871
> Project: HBase
> Issue Type: Bug
> Components: jenkins, mapreduce
>Reporter: Duo Zhang
>Priority: Major
>
> After fixing the logging problem in HBASE-26870, we could see the actual
> error.
> {noformat}
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
> --config
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
> jar
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
> importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
> test:example example/ -libjars
> /home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
> WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
> Exception in thread "main" java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.mapreduce.Driver
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (HBASE-26871) Client integration test fails on branch-2.x with ClassNotFoundException
Duo Zhang created HBASE-26871:
-
Summary: Client integration test fails on branch-2.x with
ClassNotFoundException
Key: HBASE-26871
URL: https://issues.apache.org/jira/browse/HBASE-26871
Project: HBase
Issue Type: Bug
Components: jenkins, mapreduce
Reporter: Duo Zhang
After fixing the logging problem in HBASE-26870, we could see the actual error.
{noformat}
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
--config
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
jar
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
importtsv
-Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
test:example example/ -libjars
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.mapreduce.Driver
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26870) Log4j2 integration is incorrect in nighly's client integration test
[
https://issues.apache.org/jira/browse/HBASE-26870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509463#comment-17509463
]
Duo Zhang commented on HBASE-26870:
---
OK, at least we could see what is the real problem now...
https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870-branch-2/2/artifact/output-integration/hadoop-3/mr-importtsv.err/*view*/
{noformat}
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hadoop-3/bin/hadoop
--config
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/output-integration/hadoop-3/hbase-conf/
jar
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar
importtsv
-Dimporttsv.columns=HBASE_ROW_KEY,family1:column1,family1:column4,family1:column3
test:example example/ -libjars
/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/shaded-clients/hbase-shaded-mapreduce-2.6.0-SNAPSHOT.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/audience-annotations-0.5.0.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/commons-logging-1.2.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/htrace-core4-4.1.0-incubating.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jcl-over-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/jul-to-slf4j-1.7.33.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-api-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-context-1.0.1.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/opentelemetry-semconv-1.0.1-alpha.jar:/home/jenkins/jenkins-home/workspace/ase_Nightly_HBASE-26870-branch-2/hbase-client/bin/../lib/client-facing-thirdparty/slf4j-api-1.7.33.jar
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
Exception in thread "main" java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.mapreduce.Driver
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.util.RunJar.run(RunJar.java:311)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
{noformat}
> Log4j2 integration is incorrect in nighly's client integration test
> ---
>
> Key: HBASE-26870
> URL: https://issues.apache.org/jira/browse/HBASE-26870
> Project: HBase
> Issue Type: Bug
> Components: jenkins, logging
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> On master it still uses log4j2.xml and on branch-2.x we completely misses
> this part...
> Let me prepare PRs to fix them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4243: HBASE-26812 ShortCircuitingClusterConnection fails to close RegionSca…
Apache-HBase commented on pull request #4243: URL: https://github.com/apache/hbase/pull/4243#issuecomment-1073258329 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 44s | branch-2 passed | | +1 :green_heart: | compile | 3m 21s | branch-2 passed | | +1 :green_heart: | checkstyle | 1m 2s | branch-2 passed | | +1 :green_heart: | spotbugs | 2m 35s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 37s | the patch passed | | +1 :green_heart: | compile | 3m 42s | the patch passed | | +1 :green_heart: | javac | 3m 42s | the patch passed | | +1 :green_heart: | checkstyle | 0m 22s | hbase-client: The patch generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1) | | -0 :warning: | checkstyle | 0m 46s | hbase-server: The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 7m 59s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 3m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 36m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4243 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d6ad39cd88af 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 60 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/5/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26870) Log4j2 integration is incorrect in nighly's client integration test
[
https://issues.apache.org/jira/browse/HBASE-26870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509462#comment-17509462
]
Hudson commented on HBASE-26870:
Results for branch HBASE-26870
[build #2 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870/2/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Log4j2 integration is incorrect in nighly's client integration test
> ---
>
> Key: HBASE-26870
> URL: https://issues.apache.org/jira/browse/HBASE-26870
> Project: HBase
> Issue Type: Bug
> Components: jenkins, logging
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> On master it still uses log4j2.xml and on branch-2.x we completely misses
> this part...
> Let me prepare PRs to fix them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4248: HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions
Apache-HBase commented on pull request #4248: URL: https://github.com/apache/hbase/pull/4248#issuecomment-1073256328 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | shadedjars | 5m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 24s | the patch passed | | +1 :green_heart: | compile | 1m 19s | the patch passed | | +1 :green_heart: | javac | 1m 19s | the patch passed | | +1 :green_heart: | shadedjars | 5m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 54s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 299m 17s | hbase-server in the patch passed. | | | | 326m 21s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4248 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3ba32536c91c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/testReport/ | | Max. process+thread count | 2265 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Assignee: chenglei
Status: Patch Available (was: Open)
> RSRpcServices.scan should deep clone cells when RpcCallContext is null
> --
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 2.4.11, 3.0.0-alpha-2
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
>
> When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, similar as HBASE-26036, and at the
> same time, if {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return
> partial results, it does not invoke {{RegionScannerImpl.shipped}} to release
> the used resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think this {{RSRpcServices.scan}} problem should fix for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description:
When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, similar as HBASE-26036, and at the same time, if
{{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial results,
it does not invoke {{RegionScannerImpl.shipped}} to release the used resources
such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think this {{RSRpcServices.scan}} problem should fix for future
maintainability.
was:
When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, similar as HBASE-26036, and at the same time, if
{{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial results,
it does not invoke {{RegionScannerImpl.shipped}} to release the used resources
such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
> RSRpcServices.scan should deep clone cells when RpcCallContext is null
> --
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, similar as HBASE-26036, and at the
> same time, if {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return
> partial results, it does not invoke {{RegionScannerImpl.shipped}} to release
> the used resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think this {{RSRpcServices.scan}} problem should fix for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509457#comment-17509457
]
chenglei edited comment on HBASE-26812 at 3/20/22, 1:33 PM:
[~larsh], I have open a jira HBASE-26869 to fix the problem that when there is
no {{RpcCallContext}} , {{RSRpcServices.scan}} does not deep clone the result
cells and the {{RegionScanner}} is not shipped when {{RSRpcServices.scan}}
return partial results. No matter {{ShortCircuitingClusterConnection}} should
be removed or not, I think this {{RSRpcServices.scan}} problem should fix for
future maintainability.
But for the {{User}} problem your mentioned, seems that it is hard to fix it
because when the {{HTable}} invokes {{RSRpcServices.scan}} and
{{RSRpcServices.multi}}, they are in threadPool, so there is no
{{RpcCallContext}}，let alone {{User}}. If {{User}} is so important, removing
{{ShortCircuitingClusterConnection}} seems is a more acceptable way.
was (Author: comnetwork):
[~larsh], I have open a jira HBASE-26869 to fix the problem that when there is
no {{RpcCallContext}} , {{RSRpcServices.scan}} does not deep clone the result
cells and the {{RegionScanner}} is not shipped when {{RSRpcServices.scan}}
return partial results. No matter {{ShortCircuitingClusterConnection}} should
be removed or not, I think we should fix this {{RSRpcServices.scan}} problem
for future maintainability.
But for the {{User}} problem your mentioned, seems that it is hard to fix it
because when the {{HTable}} invokes {{RSRpcServices.scan}} and
{{RSRpcServices.multi}}, they are in threadPool, so there is no
{{RpcCallContext}}，let alone {{User}}. If {{User}} is so important, removing
{{ShortCircuitingClusterConnection}} seems is a more acceptable way.
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26870) Log4j2 integration is incorrect in nighly's client integration test
[
https://issues.apache.org/jira/browse/HBASE-26870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509458#comment-17509458
]
Hudson commented on HBASE-26870:
Results for branch HBASE-26870
[build #1 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870/1/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-26870/1//console].
> Log4j2 integration is incorrect in nighly's client integration test
> ---
>
> Key: HBASE-26870
> URL: https://issues.apache.org/jira/browse/HBASE-26870
> Project: HBase
> Issue Type: Bug
> Components: jenkins, logging
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Critical
> Fix For: 2.5.0, 3.0.0-alpha-3
>
>
> On master it still uses log4j2.xml and on branch-2.x we completely misses
> this part...
> Let me prepare PRs to fix them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26812) ShortCircuitingClusterConnection fails to close RegionScanners when making short-circuited calls
[
https://issues.apache.org/jira/browse/HBASE-26812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509457#comment-17509457
]
chenglei commented on HBASE-26812:
--
[~larsh], I have open a jira HBASE-26869 to fix the problem that when there is
no {{RpcCallContext}} , {{RSRpcServices.scan}} does not deep clone the result
cells and the {{RegionScanner}} is not shipped when {{RSRpcServices.scan}}
return partial results. No matter {{ShortCircuitingClusterConnection}} should
be removed or not, I think we should fix this {{RSRpcServices.scan}} problem
for future maintainability.
But for the {{User}} problem your mentioned, seems that it is hard to fix it
because when the {{HTable}} invokes {{RSRpcServices.scan}} and
{{RSRpcServices.multi}}, they are in threadPool, so there is no
{{RpcCallContext}}，let alone {{User}}. If {{User}} is so important, removing
{{ShortCircuitingClusterConnection}} seems is a more acceptable way.
> ShortCircuitingClusterConnection fails to close RegionScanners when making
> short-circuited calls
>
>
> Key: HBASE-26812
> URL: https://issues.apache.org/jira/browse/HBASE-26812
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.4.9
>Reporter: Lars Hofhansl
>Priority: Critical
>
> Just ran into this on the Phoenix side.
> We retrieve a Connection via
> {{{}RegionCoprocessorEnvironment.createConnection... getTable(...){}}}. And
> then call get on that table. The Get's key happens to be local. Now each call
> to table.get() leaves an open StoreScanner around forever. (verified with a
> memory profiler).
> There references are held via
> RegionScannerImpl.storeHeap.scannersForDelayedClose. Eventially the
> RegionServer goes into a GC of death and can only ended with kill -9.
> The reason appears to be that in this case there is no currentCall context.
> Some time in 2.x the Rpc handler/call was made responsible for closing open
> region scanners, but we forgot to handle {{ShortCircuitingClusterConnection}}
> It's not immediately clear how to fix this. But it does make
> ShortCircuitingClusterConnection useless and dangerous. If you use it, you
> *will* create a giant memory leak.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4249: HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo…
Apache-HBase commented on pull request #4249: URL: https://github.com/apache/hbase/pull/4249#issuecomment-1073250841 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 54s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 24s | master passed | | +1 :green_heart: | compile | 3m 5s | master passed | | +1 :green_heart: | checkstyle | 0m 53s | master passed | | +1 :green_heart: | spotbugs | 2m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 3m 1s | the patch passed | | +1 :green_heart: | javac | 3m 1s | the patch passed | | -0 :warning: | checkstyle | 0m 41s | hbase-server: The patch generated 2 new + 165 unchanged - 0 fixed = 167 total (was 165) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 28s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 40m 10s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4249 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6db4fa50a634 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4249/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26870) Log4j2 integration is incorrect in nighly's client integration test
Duo Zhang created HBASE-26870: - Summary: Log4j2 integration is incorrect in nighly's client integration test Key: HBASE-26870 URL: https://issues.apache.org/jira/browse/HBASE-26870 Project: HBase Issue Type: Bug Components: jenkins, logging Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 2.5.0, 3.0.0-alpha-3 On master it still uses log4j2.xml and on branch-2.x we completely misses this part... Let me prepare PRs to fix them. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26868) Introduce read/write request size metrics
[ https://issues.apache.org/jira/browse/HBASE-26868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509453#comment-17509453 ] Bryan Beaudreault commented on HBASE-26868: --- Curious what your approach here will be? I've long been wishing for simply a bytes counter, like we currently have requests counter. > Introduce read/write request size metrics > - > > Key: HBASE-26868 > URL: https://issues.apache.org/jira/browse/HBASE-26868 > Project: HBase > Issue Type: Sub-task > Components: regionserver >Affects Versions: 2.4.11 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 > > > Calculating the read and write size(unit is bytes) in the region, and > aggregated in table and regionserver levels. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (HBASE-26813) Remove javax.ws.rs-api dependency
[
https://issues.apache.org/jira/browse/HBASE-26813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509452#comment-17509452
]
Duo Zhang commented on HBASE-26813:
---
BTW I've deleted several builds of flaky tests and master nightly avoid adding
bunch of UTs to flaky list...
> Remove javax.ws.rs-api dependency
> -
>
> Key: HBASE-26813
> URL: https://issues.apache.org/jira/browse/HBASE-26813
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> I see that we still have {{javax.ws.rs-api}} as a dependency in our
> hbase-http pom. If, for example, the {{ClientBuilder}} from this jar is used,
> it'll instantiate whatever non-shaded jersey client is on the class path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] comnetwork opened a new pull request #4249: HBASE-26869 RSRpcServices.scan should deep clone cells when RpcCallCo…
comnetwork opened a new pull request #4249: URL: https://github.com/apache/hbase/pull/4249 …ntext is null -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description:
When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, similar as HBASE-26036, and at the same time, if
{{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial results,
it does not invoke {{RegionScannerImpl.shipped}} to release the used resources
such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
was:
When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, and at the same time, if {{RpcCallContext}} is
null, when {{RSRpcServices.scan}} return partial results, it does not invoke
{{RegionScannerImpl.shipped}} to release the used resources such as
{{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
> RSRpcServices.scan should deep clone cells when RpcCallContext is null
> --
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, similar as HBASE-26036, and at the
> same time, if {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return
> partial results, it does not invoke {{RegionScannerImpl.shipped}} to release
> the used resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think we should fix this {{RSRpcServices.scan}} problem for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Summary: RSRpcServices.scan should deep clone cells when RpcCallContext is
null (was: RSRpcServices.scan should deep clone cells when RpcCallContext is
null.)
> RSRpcServices.scan should deep clone cells when RpcCallContext is null
> --
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, and at the same time, if
> {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial
> results, it does not invoke {{RegionScannerImpl.shipped}} to release the used
> resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think we should fix this {{RSRpcServices.scan}} problem for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null.
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description:
When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, and at the same time, if {{RpcCallContext}} is
null, when {{RSRpcServices.scan}} return partial results, it does not invoke
{{RegionScannerImpl.shipped}} to release the used resources such as
{{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
was:
For HBASAE-26812, I found that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, and at the same time, if {{RpcCallContext}} is
null, when {{RSRpcServices.scan}} return partial results, it does not invoke
{{RegionScannerImpl.shipped}} to release the used resources such as
{{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
> RSRpcServices.scan should deep clone cells when RpcCallContext is null.
> ---
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> When inspect HBASAE-26812, I find that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, and at the same time, if
> {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial
> results, it does not invoke {{RegionScannerImpl.shipped}} to release the used
> resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think we should fix this {{RSRpcServices.scan}} problem for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null.
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description:
For HBASAE-26812, I found that if {{RpcCallContext}} is null,
{{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
closes {{RegionScannerImpl}}, but it does not deep clone the result cells , so
these cells may be returned to the {{ByteBuffAllocator}} and may be overwritten
before the caller reads them, and at the same time, if {{RpcCallContext}} is
null, when {{RSRpcServices.scan}} return partial results, it does not invoke
{{RegionScannerImpl.shipped}} to release the used resources such as
{{HFileScanner}}, pooled {{ByteBuffer}} etc.
No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
think we should fix this {{RSRpcServices.scan}} problem for future
maintainability.
was:For HBASAE-26812, I found that when RpcCallContext is null, {{ does not
deep clone the cells and the RegionScanner is not shipped when there is no
RpcContext.
> RSRpcServices.scan should deep clone cells when RpcCallContext is null.
> ---
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> For HBASAE-26812, I found that if {{RpcCallContext}} is null,
> {{RSRpcServices.scan}} does not set {{ServerCall.rpcCallback}} and directly
> closes {{RegionScannerImpl}}, but it does not deep clone the result cells ,
> so these cells may be returned to the {{ByteBuffAllocator}} and may be
> overwritten before the caller reads them, and at the same time, if
> {{RpcCallContext}} is null, when {{RSRpcServices.scan}} return partial
> results, it does not invoke {{RegionScannerImpl.shipped}} to release the used
> resources such as {{HFileScanner}}, pooled {{ByteBuffer}} etc.
> No matter {{ShortCircuitingClusterConnection}} should be removed or not, I
> think we should fix this {{RSRpcServices.scan}} problem for future
> maintainability.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null.
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description: For HBASAE-26812, I found that when RpcCallContext is null, {{
does not deep clone the cells and the RegionScanner is not shipped when there
is no RpcContext. (was: For HBASAE-26812, I found that when RpcCallContext is
null, {{scan does not deep clone the cells and the RegionScanner is not shipped
when there is no RpcContext.)
> RSRpcServices.scan should deep clone cells when RpcCallContext is null.
> ---
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> For HBASAE-26812, I found that when RpcCallContext is null, {{ does not deep
> clone the cells and the RegionScanner is not shipped when there is no
> RpcContext.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null.
[
https://issues.apache.org/jira/browse/HBASE-26869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated HBASE-26869:
-
Description: For HBASAE-26812, I found that when RpcCallContext is null,
{{scan does not deep clone the cells and the RegionScanner is not shipped when
there is no RpcContext.
> RSRpcServices.scan should deep clone cells when RpcCallContext is null.
> ---
>
> Key: HBASE-26869
> URL: https://issues.apache.org/jira/browse/HBASE-26869
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 3.0.0-alpha-2, 2.4.11
>Reporter: chenglei
>Priority: Major
>
> For HBASAE-26812, I found that when RpcCallContext is null, {{scan does not
> deep clone the cells and the RegionScanner is not shipped when there is no
> RpcContext.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4248: HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions
Apache-HBase commented on pull request #4248: URL: https://github.com/apache/hbase/pull/4248#issuecomment-1073232826 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 58s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 51s | master passed | | +1 :green_heart: | compile | 1m 0s | master passed | | +1 :green_heart: | shadedjars | 3m 42s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 1m 0s | the patch passed | | +1 :green_heart: | javac | 1m 0s | the patch passed | | +1 :green_heart: | shadedjars | 3m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 38s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 177m 26s | hbase-server in the patch passed. | | | | 197m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4248 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 3712b5c0fb7c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/testReport/ | | Max. process+thread count | 2497 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26869) RSRpcServices.scan should deep clone cells when RpcCallContext is null.
chenglei created HBASE-26869: Summary: RSRpcServices.scan should deep clone cells when RpcCallContext is null. Key: HBASE-26869 URL: https://issues.apache.org/jira/browse/HBASE-26869 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 2.4.11, 3.0.0-alpha-2 Reporter: chenglei -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4246: HBASE-26867 Introduce a FlushProcedure
Apache-HBase commented on pull request #4246: URL: https://github.com/apache/hbase/pull/4246#issuecomment-1073228087 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 14s | master passed | | +1 :green_heart: | compile | 2m 42s | master passed | | +1 :green_heart: | shadedjars | 5m 27s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 17s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 48s | the patch passed | | +1 :green_heart: | compile | 2m 37s | the patch passed | | +1 :green_heart: | javac | 2m 37s | the patch passed | | -1 :x: | shadedjars | 4m 29s | patch has 10 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 56s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 2m 3s | hbase-client in the patch passed. | | -1 :x: | unit | 301m 1s | hbase-server in the patch failed. | | | | 334m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4246 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 233b7fc1b59a 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-11.0.10+9 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/testReport/ | | Max. process+thread count | 2473 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4247: HBASE-26862 Fix CME when closing region
Apache-HBase commented on pull request #4247: URL: https://github.com/apache/hbase/pull/4247#issuecomment-1073217572 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 2s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 4m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 21s | the patch passed | | +1 :green_heart: | compile | 0m 50s | the patch passed | | +1 :green_heart: | javac | 0m 50s | the patch passed | | +1 :green_heart: | shadedjars | 4m 13s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 178m 51s | hbase-server in the patch passed. | | | | 200m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4247 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bffd12290590 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/testReport/ | | Max. process+thread count | 3318 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4247: HBASE-26862 Fix CME when closing region
Apache-HBase commented on pull request #4247: URL: https://github.com/apache/hbase/pull/4247#issuecomment-1073217430 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 41s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 0m 39s | master passed | | +1 :green_heart: | shadedjars | 4m 4s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 27s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 25s | the patch passed | | +1 :green_heart: | compile | 0m 38s | the patch passed | | +1 :green_heart: | javac | 0m 38s | the patch passed | | +1 :green_heart: | shadedjars | 4m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 181m 43s | hbase-server in the patch passed. | | | | 199m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4247 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0feb89d063c0 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/testReport/ | | Max. process+thread count | 2533 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26813) Remove javax.ws.rs-api dependency
[
https://issues.apache.org/jira/browse/HBASE-26813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509402#comment-17509402
]
Hudson commented on HBASE-26813:
Results for branch branch-2
[build #491 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/491//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Remove javax.ws.rs-api dependency
> -
>
> Key: HBASE-26813
> URL: https://issues.apache.org/jira/browse/HBASE-26813
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 2.5.0, 2.6.0, 3.0.0-alpha-3
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> I see that we still have {{javax.ws.rs-api}} as a dependency in our
> hbase-http pom. If, for example, the {{ClientBuilder}} from this jar is used,
> it'll instantiate whatever non-shaded jersey client is on the class path.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4248: HBASE-26175 MetricsHBaseServer should record all kinds of Exceptions
Apache-HBase commented on pull request #4248: URL: https://github.com/apache/hbase/pull/4248#issuecomment-1073202570 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 15s | master passed | | +1 :green_heart: | compile | 2m 29s | master passed | | +1 :green_heart: | checkstyle | 0m 42s | master passed | | +1 :green_heart: | spotbugs | 1m 33s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 2m 27s | the patch passed | | +1 :green_heart: | javac | 2m 27s | the patch passed | | +1 :green_heart: | checkstyle | 0m 7s | The patch passed checkstyle in hbase-hadoop-compat | | +1 :green_heart: | checkstyle | 0m 33s | hbase-server: The patch generated 0 new + 12 unchanged - 1 fixed = 12 total (was 13) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 6m 54s | The patch causes 10 errors with Hadoop v3.2.2. | | +1 :green_heart: | spotbugs | 1m 43s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 27m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4248 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux d8c0b5c9ed2a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | hadoopcheck | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/artifact/yetus-general-check/output/patch-javac-3.2.2.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4248/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4246: HBASE-26867 Introduce a FlushProcedure
Apache-HBase commented on pull request #4246: URL: https://github.com/apache/hbase/pull/4246#issuecomment-1073202347 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 19s | master passed | | +1 :green_heart: | compile | 1m 12s | master passed | | +1 :green_heart: | shadedjars | 3m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 4s | the patch passed | | +1 :green_heart: | compile | 1m 13s | the patch passed | | +1 :green_heart: | javac | 1m 13s | the patch passed | | -1 :x: | shadedjars | 2m 57s | patch has 10 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 24s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | unit | 1m 9s | hbase-client in the patch passed. | | -1 :x: | unit | 172m 35s | hbase-server in the patch failed. | | | | 191m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4246 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 502e7f27e75a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | shadedjars | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/testReport/ | | Max. process+thread count | 2988 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #4243: HBASE-26812 ShortCircuitingClusterConnection fails to close RegionSca…
Apache-HBase commented on pull request #4243: URL: https://github.com/apache/hbase/pull/4243#issuecomment-1073200769 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 16s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 56s | branch-2 passed | | +1 :green_heart: | compile | 1m 48s | branch-2 passed | | +1 :green_heart: | shadedjars | 0m 39s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 12s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 45s | the patch passed | | +1 :green_heart: | compile | 1m 55s | the patch passed | | +1 :green_heart: | javac | 1m 55s | the patch passed | | +1 :green_heart: | shadedjars | 0m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 47s | hbase-client in the patch passed. | | -1 :x: | unit | 343m 37s | hbase-server in the patch failed. | | | | 367m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4243 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux bf7c9c580e4c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / ba14796289 | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/4/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/4/testReport/ | | Max. process+thread count | 2238 (vs. ulimit of 12500) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4243/4/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26862) Fix CME when closing region
[
https://issues.apache.org/jira/browse/HBASE-26862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26862:
---
Component/s: regionserver
> Fix CME when closing region
> ---
>
> Key: HBASE-26862
> URL: https://issues.apache.org/jira/browse/HBASE-26862
> Project: HBase
> Issue Type: Bug
> Components: regionserver, Scanners
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
>
> KeyValueHeap.scannersForDelayedClose is not thread safe, CME might occurs
> when closing a scanning region, who is adding scanners to the delayed close
> list.
> {code:java}
> 2022-03-18 17:24:17,435 INFO [regionserver/hostname/ip:16020-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> hmart-waimaiups_poi_bitmap_theme_pool,,1645769651132.56363df04da85752aa67669989de19fd.;
> java.util.ConcurrentModificationException
> java.io.IOException: java.util.ConcurrentModificationException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1767)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1593)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:394)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:276)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:567)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:156)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:221)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:548)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:520)
> at
> org.apache.hadoop.hbase.regionserver.HStore.closeChangedReadersObservers(HStore.java:1296)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:1001)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:131)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1745)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1741)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26862) Fix CME when closing region
[
https://issues.apache.org/jira/browse/HBASE-26862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26862:
---
Priority: Minor (was: Major)
> Fix CME when closing region
> ---
>
> Key: HBASE-26862
> URL: https://issues.apache.org/jira/browse/HBASE-26862
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
>
> KeyValueHeap.scannersForDelayedClose is not thread safe, CME might occurs
> when closing a scanning region, who is adding scanners to the delayed close
> list.
> {code:java}
> 2022-03-18 17:24:17,435 INFO [regionserver/hostname/ip:16020-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> hmart-waimaiups_poi_bitmap_theme_pool,,1645769651132.56363df04da85752aa67669989de19fd.;
> java.util.ConcurrentModificationException
> java.io.IOException: java.util.ConcurrentModificationException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1767)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1593)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:394)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:276)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:567)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:156)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:221)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:548)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:520)
> at
> org.apache.hadoop.hbase.regionserver.HStore.closeChangedReadersObservers(HStore.java:1296)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:1001)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:131)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1745)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1741)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26175:
---
Status: Patch Available (was: Open)
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26862) Fix CME when closing region
[
https://issues.apache.org/jira/browse/HBASE-26862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26862:
---
Status: Patch Available (was: Open)
> Fix CME when closing region
> ---
>
> Key: HBASE-26862
> URL: https://issues.apache.org/jira/browse/HBASE-26862
> Project: HBase
> Issue Type: Bug
> Components: Scanners
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
>
> KeyValueHeap.scannersForDelayedClose is not thread safe, CME might occurs
> when closing a scanning region, who is adding scanners to the delayed close
> list.
> {code:java}
> 2022-03-18 17:24:17,435 INFO [regionserver/hostname/ip:16020-splits-0]
> regionserver.SplitRequest: Running rollback/cleanup of failed split of
> hmart-waimaiups_poi_bitmap_theme_pool,,1645769651132.56363df04da85752aa67669989de19fd.;
> java.util.ConcurrentModificationException
> java.io.IOException: java.util.ConcurrentModificationException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1767)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:1593)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.stepsBeforePONR(SplitTransactionImpl.java:394)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.createDaughters(SplitTransactionImpl.java:276)
> at
> org.apache.hadoop.hbase.regionserver.SplitTransactionImpl.execute(SplitTransactionImpl.java:567)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.doSplitting(SplitRequest.java:82)
> at
> org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:156)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.ConcurrentModificationException
> at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
> at java.util.ArrayList$Itr.next(ArrayList.java:851)
> at
> org.apache.hadoop.hbase.regionserver.KeyValueHeap.close(KeyValueHeap.java:221)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:548)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.close(StoreScanner.java:520)
> at
> org.apache.hadoop.hbase.regionserver.HStore.closeChangedReadersObservers(HStore.java:1296)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:1001)
> at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:131)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1745)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1741)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha reassigned HBASE-26175:
--
Assignee: Xiaolin Ha
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26175:
---
Affects Version/s: 2.4.11
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Priority: Minor
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26175:
---
Priority: Minor (was: Major)
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Xiaolin Ha
>Priority: Minor
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha updated HBASE-26175:
---
Fix Version/s: 2.5.0
2.6.0
3.0.0-alpha-3
2.4.12
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Affects Versions: 2.4.11
>Reporter: Xiaolin Ha
>Priority: Minor
> Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3, 2.4.12
>
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HBASE-26175) MetricsHBaseServer should record all kinds of Exceptions
[
https://issues.apache.org/jira/browse/HBASE-26175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17509392#comment-17509392
]
Xiaolin Ha commented on HBASE-26175:
[~pankajkumar] Yeah, thanks, I'll add a new category for other exceptions, and
keeps the debug logs.
> MetricsHBaseServer should record all kinds of Exceptions
>
>
> Key: HBASE-26175
> URL: https://issues.apache.org/jira/browse/HBASE-26175
> Project: HBase
> Issue Type: Improvement
> Components: metrics
>Reporter: Xiaolin Ha
>Priority: Major
> Attachments: RequestTooBigException.png
>
>
> We can define a kind of Exception such as OtherExcpetions to record
> exceptions doesn't in the following kinds of exceptions. Only debug those
> exceptions by LOG.debug("Unknown exception type", throwable); is not helpful
> to find errors.
> {code:java}
> if (throwable != null) {
> if (throwable instanceof OutOfOrderScannerNextException) {
> source.outOfOrderException();
> } else if (throwable instanceof RegionTooBusyException) {
> source.tooBusyException();
> } else if (throwable instanceof UnknownScannerException) {
> source.unknownScannerException();
> } else if (throwable instanceof ScannerResetException) {
> source.scannerResetException();
> } else if (throwable instanceof RegionMovedException) {
> source.movedRegionException();
> } else if (throwable instanceof NotServingRegionException) {
> source.notServingRegionException();
> } else if (throwable instanceof FailedSanityCheckException) {
> source.failedSanityException();
> } else if (throwable instanceof MultiActionResultTooLarge) {
> source.multiActionTooLargeException();
> } else if (throwable instanceof CallQueueTooBigException) {
> source.callQueueTooBigException();
> } else if (throwable instanceof QuotaExceededException) {
> source.quotaExceededException();
> } else if (throwable instanceof RpcThrottlingException) {
> source.rpcThrottlingException();
> } else if (LOG.isDebugEnabled()) {
> LOG.debug("Unknown exception type", throwable);
> }
> }
> {code}
> !RequestTooBigException.png|width=787,height=336!
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4247: HBASE-26862 Fix CME when closing region
Apache-HBase commented on pull request #4247: URL: https://github.com/apache/hbase/pull/4247#issuecomment-1073188964 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 24s | master passed | | +1 :green_heart: | compile | 2m 47s | master passed | | +1 :green_heart: | checkstyle | 0m 42s | master passed | | +1 :green_heart: | spotbugs | 1m 34s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 49s | the patch passed | | +1 :green_heart: | compile | 2m 57s | the patch passed | | +1 :green_heart: | javac | 2m 57s | the patch passed | | +1 :green_heart: | checkstyle | 0m 42s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 16m 51s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | spotbugs | 2m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 41m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4247 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 71a921ef8271 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4247/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26868) Introduce read/write request size metrics
Xiaolin Ha created HBASE-26868: -- Summary: Introduce read/write request size metrics Key: HBASE-26868 URL: https://issues.apache.org/jira/browse/HBASE-26868 Project: HBase Issue Type: Sub-task Components: regionserver Affects Versions: 2.4.11 Reporter: Xiaolin Ha Assignee: Xiaolin Ha Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3 Calculating the read and write size(unit is bytes) in the region, and aggregated in table and regionserver levels. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [hbase] Apache-HBase commented on pull request #4246: HBASE-26867 Introduce a FlushProcedure
Apache-HBase commented on pull request #4246: URL: https://github.com/apache/hbase/pull/4246#issuecomment-1073177533 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 57s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 17s | master passed | | +1 :green_heart: | compile | 3m 47s | master passed | | +1 :green_heart: | checkstyle | 1m 4s | master passed | | +1 :green_heart: | spotbugs | 4m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 6s | the patch passed | | +1 :green_heart: | compile | 3m 46s | the patch passed | | +1 :green_heart: | cc | 3m 46s | the patch passed | | +1 :green_heart: | javac | 3m 46s | the patch passed | | -0 :warning: | checkstyle | 0m 18s | hbase-client: The patch generated 6 new + 68 unchanged - 0 fixed = 74 total (was 68) | | -0 :warning: | checkstyle | 0m 37s | hbase-server: The patch generated 22 new + 188 unchanged - 0 fixed = 210 total (was 188) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 26s | Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1. | | +1 :green_heart: | hbaseprotoc | 1m 20s | the patch passed | | +1 :green_heart: | spotbugs | 4m 51s | the patch passed | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 27s | The patch generated 2 ASF License warnings. | | | | 43m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/4246 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux cc1a8522b003 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 53a7ca794a | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | asflicense | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 60 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4246/1/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
