[GitHub] [hbase] chrajeshbabu merged pull request #5176: HBASE-27792 Guard Master/RS Dump Servlet behind admin walls
chrajeshbabu merged PR #5176: URL: https://github.com/apache/hbase/pull/5176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27109) Move replication queue storage from zookeeper to a separated HBase table
[
https://issues.apache.org/jira/browse/HBASE-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716060#comment-17716060
]
Hudson commented on HBASE-27109:
Results for branch HBASE-27109/table_based_rqs
[build #64 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/64/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/64/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/64/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/64/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move replication queue storage from zookeeper to a separated HBase table
>
>
> Key: HBASE-27109
> URL: https://issues.apache.org/jira/browse/HBASE-27109
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> This is a more specific issue based on the works which are already done in
> HBASE-15867.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] bbeaudreault commented on a diff in pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
bbeaudreault commented on code in PR #5171:
URL: https://github.com/apache/hbase/pull/5171#discussion_r1175853711
##
hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparatorImpl.java:
##
@@ -448,6 +451,33 @@ public final int compareFamilies(Cell left, Cell right) {
right.getFamilyArray(), right.getFamilyOffset(),
right.getFamilyLength());
}
+ /**
+ * This method will be overridden when we compare cells inner store to
bypass family comparing.
+ */
+ protected int compareFamilies(KeyValue left, KeyValue right, int
leftFamilyLength,
Review Comment:
Can you use more similar signature to existing methods? I find this ordering
confusing, because everyone is used to doing left, leftOffset, leftLength,
right, rightOffset, rightLenght.
This goes for all the new methods.
##
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:
##
@@ -277,14 +277,6 @@ public class HRegion implements HeapSize,
PropagatingConfigurationObserver, Regi
public static final String RECOVERED_EDITS_IGNORE_EOF =
"hbase.hregion.recovered.edits.ignore.eof";
- /**
- * Whether to use {@link MetaCellComparator} even if we are not meta region.
Used when creating
- * master local region.
- */
- public static final String USE_META_CELL_COMPARATOR =
"hbase.region.use.meta.cell.comparator";
Review Comment:
Not a huge deal, but why move this constant in this PR? Doesn't seem related?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-27798) Client side should back off based on wait interval in RpcThrottlingException
[ https://issues.apache.org/jira/browse/HBASE-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715954#comment-17715954 ] Ray Mattingly commented on HBASE-27798: --- sounds good, thanks for the input! > Client side should back off based on wait interval in RpcThrottlingException > > > Key: HBASE-27798 > URL: https://issues.apache.org/jira/browse/HBASE-27798 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27798) Client side should back off based on wait interval in RpcThrottlingException
[ https://issues.apache.org/jira/browse/HBASE-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715953#comment-17715953 ] Bryan Beaudreault commented on HBASE-27798: --- I don’t think so. In this case the throttling exception has specific context about when the retry should happen based on the state of the limiter. Seems like we should just trust that as a first pass. Maybe we can add in one or the other component in a followup Jiraif for whatever reason we find that the retries are happening too quickly. > Client side should back off based on wait interval in RpcThrottlingException > > > Key: HBASE-27798 > URL: https://issues.apache.org/jira/browse/HBASE-27798 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5200: HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (addendum)
Apache-HBase commented on PR #5200: URL: https://github.com/apache/hbase/pull/5200#issuecomment-1520694790 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 3s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 45s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 399m 43s | root in the patch passed. | | | | 412m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5200 | | Optional Tests | unit | | uname | Linux 40a2c4a07f9d 5.4.0-1097-aws #105~18.04.1-Ubuntu SMP Mon Feb 13 17:50:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 95f83cb8e2 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/testReport/ | | Max. process+thread count | 5093 (vs. ulimit of 3) | | modules | C: hbase-logging . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27798) Client side should back off based on wait interval in RpcThrottlingException
[ https://issues.apache.org/jira/browse/HBASE-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715947#comment-17715947 ] Ray Mattingly commented on HBASE-27798: --- That sounds good to me. Should we still add the retry backoff for the waitInterval case? > Client side should back off based on wait interval in RpcThrottlingException > > > Key: HBASE-27798 > URL: https://issues.apache.org/jira/browse/HBASE-27798 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27798) Client side should back off based on wait interval in RpcThrottlingException
[ https://issues.apache.org/jira/browse/HBASE-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715943#comment-17715943 ] Bryan Beaudreault commented on HBASE-27798: --- I was thinking it would be a replacement > Client side should back off based on wait interval in RpcThrottlingException > > > Key: HBASE-27798 > URL: https://issues.apache.org/jira/browse/HBASE-27798 > Project: HBase > Issue Type: Improvement >Reporter: Bryan Beaudreault >Assignee: Ray Mattingly >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27798) Client side should back off based on wait interval in RpcThrottlingException
[
https://issues.apache.org/jira/browse/HBASE-27798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715939#comment-17715939
]
Ray Mattingly commented on HBASE-27798:
---
The current retry backoff has a few inputs & steps. For example, we use [the
{{pause}} and {{pauseForServerOverloaded}} durations in
{{RpcRetryingCallerImpl}}|https://github.com/apache/hbase/blob/branch-2/hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerImpl.java#L59-L60]
to determine a pauseBase millis, and then multiply it by the relevant
{{RETRY_BACKOFF}} item.
I'm wondering how we should incorporate the {{waitInterval}} into this existing
system; I see a few options:
# We could consider waitInterval an addition to the pauseBase
# We could consider waitInterval an addition to the product of pauseBase *
retryBackoff
# We could consider waitInterval, if present, to be a replacement for the
pauseBase
# We could consider waitInterval, if present, to be a replacement for the
produce of pauseBase * retryBackoff
[~bbeaudreault] do you have any thoughts/preference here?
> Client side should back off based on wait interval in RpcThrottlingException
>
>
> Key: HBASE-27798
> URL: https://issues.apache.org/jira/browse/HBASE-27798
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Ray Mattingly
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27806) Support dynamic reinitializing replication peer storage
[
https://issues.apache.org/jira/browse/HBASE-27806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715925#comment-17715925
]
Hudson commented on HBASE-27806:
Results for branch master
[build #824 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Support dynamic reinitializing replication peer storage
> ---
>
> Key: HBASE-27806
> URL: https://issues.apache.org/jira/browse/HBASE-27806
> Project: HBase
> Issue Type: Sub-task
> Components: conf, Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> When writing the documentation about how to migrate replication data online,
> I found out that, the replication related classes do not implement
> ConfigurationObserver, so we can not use online configuration update to load
> new replication peer storage...
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-20804) Document and add tests for HBaseConfTool
[
https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715926#comment-17715926
]
Hudson commented on HBASE-20804:
Results for branch master
[build #824 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document and add tests for HBaseConfTool
>
>
> Key: HBASE-20804
> URL: https://issues.apache.org/jira/browse/HBASE-20804
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, tooling
>Reporter: Andy Lin
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18
>
> Attachments: HBASE-20804.master.001.patch,
> HBASE-20804.master.002.patch, HBASE-20804.master.002.patch
>
>
> Add documentation and tests for HBaseConfTool.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27807) PressureAwareCompactionThroughputController#tune log the opposite of the actual scenario
[
https://issues.apache.org/jira/browse/HBASE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715927#comment-17715927
]
Hudson commented on HBASE-27807:
Results for branch master
[build #824 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/824/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> PressureAwareCompactionThroughputController#tune log the opposite of the
> actual scenario
>
>
> Key: HBASE-27807
> URL: https://issues.apache.org/jira/browse/HBASE-27807
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 3.0.0-alpha-1, 1.5.0, 2.3.0
>Reporter: haosen chen
>Assignee: haosen chen
>Priority: Trivial
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18
>
> Attachments:
> 0001-HBASE-27807-PressureAwareCompactionThroughputControl.patch,
> image-2023-04-21-14-28-19-766.png
>
>
> In
> [https://github.com/apache/hbase/commit/07c71d630cd293ab55ac85e9bfa06033598134c4]
> , the condition of the if statement is incorrectly modified.
> !image-2023-04-21-14-28-19-766.png!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-20804) Document and add tests for HBaseConfTool
[
https://issues.apache.org/jira/browse/HBASE-20804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715917#comment-17715917
]
Hudson commented on HBASE-20804:
Results for branch branch-2.5
[build #342 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Document and add tests for HBaseConfTool
>
>
> Key: HBASE-20804
> URL: https://issues.apache.org/jira/browse/HBASE-20804
> Project: HBase
> Issue Type: Sub-task
> Components: documentation, tooling
>Reporter: Andy Lin
>Assignee: Nihal Jain
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18
>
> Attachments: HBASE-20804.master.001.patch,
> HBASE-20804.master.002.patch, HBASE-20804.master.002.patch
>
>
> Add documentation and tests for HBaseConfTool.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-27807) PressureAwareCompactionThroughputController#tune log the opposite of the actual scenario
[
https://issues.apache.org/jira/browse/HBASE-27807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715916#comment-17715916
]
Hudson commented on HBASE-27807:
Results for branch branch-2.5
[build #342 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/342/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> PressureAwareCompactionThroughputController#tune log the opposite of the
> actual scenario
>
>
> Key: HBASE-27807
> URL: https://issues.apache.org/jira/browse/HBASE-27807
> Project: HBase
> Issue Type: Bug
> Components: Compaction
>Affects Versions: 3.0.0-alpha-1, 1.5.0, 2.3.0
>Reporter: haosen chen
>Assignee: haosen chen
>Priority: Trivial
> Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5, 2.4.18
>
> Attachments:
> 0001-HBASE-27807-PressureAwareCompactionThroughputControl.patch,
> image-2023-04-21-14-28-19-766.png
>
>
> In
> [https://github.com/apache/hbase/commit/07c71d630cd293ab55ac85e9bfa06033598134c4]
> , the condition of the if statement is incorrectly modified.
> !image-2023-04-21-14-28-19-766.png!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5200: HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (addendum)
Apache-HBase commented on PR #5200: URL: https://github.com/apache/hbase/pull/5200#issuecomment-1520527730 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 266m 53s | root in the patch passed. | | | | 281m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5200 | | Optional Tests | unit | | uname | Linux b6df4b8ef593 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 95f83cb8e2 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/testReport/ | | Max. process+thread count | 4601 (vs. ulimit of 3) | | modules | C: hbase-logging . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk merged pull request #5200: HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (addendum)
ndimiduk merged PR #5200: URL: https://github.com/apache/hbase/pull/5200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #5200: HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (addendum)
Apache-HBase commented on PR #5200: URL: https://github.com/apache/hbase/pull/5200#issuecomment-1520059190 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 22s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 4s | Maven dependency ordering for patch | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 0m 35s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 3m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5200 | | Optional Tests | dupname asflicense spotless | | uname | Linux 210d345c93c1 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 95f83cb8e2 | | Max. process+thread count | 43 (vs. ulimit of 3) | | modules | C: . hbase-logging U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5200/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ndimiduk opened a new pull request, #5200: HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC (adendum)
ndimiduk opened a new pull request, #5200: URL: https://github.com/apache/hbase/pull/5200 Make use of `%notEmpty` pattern with MDC logging. Will function as expected once we upgrade to Log4J 2.21.0+. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27762) Include EventType and ProcedureV2 pid in logging via MDC
[
https://issues.apache.org/jira/browse/HBASE-27762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715745#comment-17715745
]
Nick Dimiduk commented on HBASE-27762:
--
They've got a fix on LOG4J2-3660 and I've verified that it behaves as I
expected. We'll need to upgrade to a more recent Log4J2, once it's released, in
order to get the fix.
> Include EventType and ProcedureV2 pid in logging via MDC
>
>
> Key: HBASE-27762
> URL: https://issues.apache.org/jira/browse/HBASE-27762
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.5.3, 2.4.17
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Major
> Fix For: 3.0.0-alpha-4
>
>
> Tracing the distributed actions of ProcedureV2 {{RemoteProcedure}} execution
> is painful. We are pretty good about logging the {{proc_id}} most of the
> time, but we don't catch them all. Now that we're up on log4j2, let's use the
> MDC feature to include some basic information universally.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [hbase] bsglz commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
bsglz commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519997732 Thanks a lot for your reviewing and detailed testing. @Apache9 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519982050 I tested for scenarios where 1. both empty families 2. empty family comparing to none empty family 3. same family 4. different families The performance number looks good. The inner store scenario are always better, especially when we need to compare two non empty families(of course in the test the length of the row is small, so the cost for comparing families looks expensive). And for the scenario where we can not use inner store comparator, the performance is almost the same. For me I think it is OK to let the PR go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519965013   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519948632 I updated the JHM microbenchmark. And this is the result ··· Benchmark (families) Mode Samples Score Score error Units o.a.h.h.ComparatorBenchmark.innerStoreCompareBBKV- thrpt 20 55617310.043 521836.982 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareBBKV -family thrpt 20 97491057.894 896496.011 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareBBKVfamily-family thrpt 20 55874045.457 506603.074 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareBBKVfamily-family_new thrpt 20 97321488.589 941602.585 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKV - thrpt 20 50564430.713 494912.986 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKV-family thrpt 20 102582122.364 1237571.556 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKV family-family thrpt 20 50691911.148 613201.615 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKV family-family_new thrpt 20 101956533.610 1664737.970 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKVVsBBKV- thrpt 20 56708491.949 1163512.704 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKVVsBBKV -family thrpt 20 97970699.118 1363796.393 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKVVsBBKVfamily-family thrpt 20 57059944.100 1002170.554 ops/s o.a.h.h.ComparatorBenchmark.innerStoreCompareKVVsBBKVfamily-family_new thrpt 20 98271151.309 1287525.026 ops/s o.a.h.h.ComparatorBenchmark.newCompareBBKV - thrpt 20 50337883.151 562547.715 ops/s o.a.h.h.ComparatorBenchmark.newCompareBBKV -family thrpt 20 83762676.170 1506319.700 ops/s o.a.h.h.ComparatorBenchmark.newCompareBBKV family-family thrpt 20 45054227.460 624259.638 ops/s o.a.h.h.ComparatorBenchmark.newCompareBBKV family-family_new thrpt 20 66497651.217 837933.702 ops/s o.a.h.h.ComparatorBenchmark.newCompareKV - thrpt 20 47594905.620 728110.022 ops/s o.a.h.h.ComparatorBenchmark.newCompareKV -family thrpt 20 89348636.664 983654.464 ops/s o.a.h.h.ComparatorBenchmark.newCompareKV family-family thrpt 20 42626078.523 617300.607 ops/s o.a.h.h.ComparatorBenchmark.newCompareKV family-family_new thrpt 20 58994862.559 688422.723 ops/s o.a.h.h.ComparatorBenchmark.newCompareKVVsBBKV - thrpt 20 55049124.220 1200062.501 ops/s o.a.h.h.ComparatorBenchmark.newCompareKVVsBBKV -family thrpt 20 88984928.343 2043302.430 ops/s o.a.h.h.ComparatorBenchmark.newCompareKVVsBBKV family-family thrpt 20 45682012.123 735156.659 ops/s o.a.h.h.ComparatorBenchmark.newCompareKVVsBBKV family-family_new thrpt 20 70523321.407 1360574.335 ops/s o.a.h.h.ComparatorBenchmark.oldCompareBBKV - thrpt 20 49687744.273 692599.695 ops/s o.a.h.h.ComparatorBenchmark.oldCompareBBKV -family thrpt 20 83617950.799 1316833.712 ops/s o.a.h.h.ComparatorBenchmark.oldCompareBBKV family-family thrpt 20 8877.187 495430.522 ops/s o.a.h.h.ComparatorBenchmark.oldCompareBBKV family-family_new thrpt 20 65398995.122 834301.661 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKV - thrpt 20 46026073.342 1128303.844 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKV -family thrpt 20 86778474.382 1235433.965 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKV family-family thrpt 20 42411664.653 697620.411 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKV family-family_new thrpt 20 58466169.799 471078.411 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKVVsBBKV - thrpt 20 54059920.896 615174.961 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKVVsBBKV -family thrpt 20 94566477.269 1704120.360 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKVVsBBKV family-family thrpt 20 46869352.874 785310.795 ops/s o.a.h.h.ComparatorBenchmark.oldCompareKVVsBBKV family-family_new thrpt 20 71263491.290 684005.539 ops/s ···
[GitHub] [hbase] Apache-HBase commented on pull request #5193: HBASE-27805 The chunk created by mslab may cause memory fragement and…
Apache-HBase commented on PR #5193: URL: https://github.com/apache/hbase/pull/5193#issuecomment-1519888047 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 21s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 4s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 52s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 10s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 47s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | +1 :green_heart: | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 390m 6s | root in the patch passed. | | | | 418m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5193 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 4c2712c26b3a 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 95f83cb8e2 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/testReport/ | | Max. process+thread count | 4798 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-27572) resource leak for store flush during high load
[ https://issues.apache.org/jira/browse/HBASE-27572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715681#comment-17715681 ] Antonio Giannone commented on HBASE-27572: -- Hi all, I report the same behaviour detailed in the description section of this issue. We are using HBase 2.4.13 and SASL authentication. To improve cluster stability, in our production environment we switch from the asnychfswal provider to the filesystem one. > resource leak for store flush during high load > -- > > Key: HBASE-27572 > URL: https://issues.apache.org/jira/browse/HBASE-27572 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 2.4.15 >Reporter: Mariusz Szpatuśko >Priority: Blocker > > 2023-01-17 14:01:00,294 WARN [Close-WAL-Writer-3] > asyncfs.FanOutOneBlockAsyncDFSOutputHelper: complete file > /hbase/WALs/prod-storage-4,16020,1673962204907/prod-storage-4%2C16020%2C1673962204907.1673964006883 > not finished, retry = 0 > 2023-01-17 14:08:02,681 ERROR > [regionserver/prod-storage-4:16020-shortCompactions-0] > util.ResourceLeakDetector: iter(StoreFlusher.java:68) > > org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75) > org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:992) > > org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2358) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2837) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2579) > > org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2551) > org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2421) > > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:588) > > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:558) > > org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:64) > > org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:340) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] Apache-HBase commented on pull request #5193: HBASE-27805 The chunk created by mslab may cause memory fragement and…
Apache-HBase commented on PR #5193: URL: https://github.com/apache/hbase/pull/5193#issuecomment-1519672233 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 23s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 49s | master passed | | +1 :green_heart: | compile | 2m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 39s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 2s | the patch passed | | +1 :green_heart: | compile | 1m 53s | the patch passed | | +1 :green_heart: | javac | 1m 53s | the patch passed | | +1 :green_heart: | shadedjars | 4m 21s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 285m 24s | root in the patch passed. | | | | 318m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5193 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux db7267f512f2 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 95f83cb8e2 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/testReport/ | | Max. process+thread count | 4611 (vs. ulimit of 3) | | modules | C: hbase-common hbase-server . U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5193/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
bsglz commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519635788 The params of BenchmarkForInnerStore.java and BenchmarkForNormal.java are different. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519633417 I just copied your code... And I think we should also use black hole to eat the return value so JVM will not optimize it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
bsglz commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519632795 Tested again with newest master code: Benchmark | (p1) | (p2) | Mode | nt | Score | Units | Diff -- | -- | -- | -- | -- | -- | -- | -- BenchmarkForInnerStore.new_compareBBKV | | | thrpt | 5 | 28197934 | ops/s | 2.00% BenchmarkForInnerStore.new_compareBBKV | | fam1 | thrpt | 5 | 45919384 | ops/s | 14.00% BenchmarkForInnerStore.new_compareBBKV | fam1 | | thrpt | 5 | 46159045 | ops/s | 15.00% BenchmarkForInnerStore.new_compareBBKV | fam1 | fam1 | thrpt | 5 | 28364753 | ops/s | 20.00% BenchmarkForInnerStore.new_compareKV | | | thrpt | 5 | 28486726 | ops/s | 1.00% BenchmarkForInnerStore.new_compareKV | | fam1 | thrpt | 5 | 48143374 | ops/s | 11.00% BenchmarkForInnerStore.new_compareKV | fam1 | | thrpt | 5 | 48582024 | ops/s | 12.00% BenchmarkForInnerStore.new_compareKV | fam1 | fam1 | thrpt | 5 | 28377232 | ops/s | 27.00% BenchmarkForInnerStore.new_compareKVVsBBKV | | | thrpt | 5 | 28477347 | ops/s | 3.00% BenchmarkForInnerStore.new_compareKVVsBBKV | | fam1 | thrpt | 5 | 47035467 | ops/s | 7.00% BenchmarkForInnerStore.new_compareKVVsBBKV | fam1 | | thrpt | 5 | 46981953 | ops/s | 7.00% BenchmarkForInnerStore.new_compareKVVsBBKV | fam1 | fam1 | thrpt | 5 | 28671804 | ops/s | 17.00% BenchmarkForInnerStore.old_compareBBKV | | | thrpt | 5 | 27696746 | ops/s | BenchmarkForInnerStore.old_compareBBKV | | fam1 | thrpt | 5 | 40146613 | ops/s | BenchmarkForInnerStore.old_compareBBKV | fam1 | | thrpt | 5 | 40048909 | ops/s | BenchmarkForInnerStore.old_compareBBKV | fam1 | fam1 | thrpt | 5 | 23722820 | ops/s | BenchmarkForInnerStore.old_compareKV | | | thrpt | 5 | 28192740 | ops/s | BenchmarkForInnerStore.old_compareKV | | fam1 | thrpt | 5 | 43224450 | ops/s | BenchmarkForInnerStore.old_compareKV | fam1 | | thrpt | 5 | 43351477 | ops/s | BenchmarkForInnerStore.old_compareKV | fam1 | fam1 | thrpt | 5 | 22412038 | ops/s | BenchmarkForInnerStore.old_compareKVVsBBKV | | | thrpt | 5 | 27559904 | ops/s | BenchmarkForInnerStore.old_compareKVVsBBKV | | fam1 | thrpt | 5 | 43857122 | ops/s | BenchmarkForInnerStore.old_compareKVVsBBKV | fam1 | | thrpt | 5 | 43884642 | ops/s | BenchmarkForInnerStore.old_compareKVVsBBKV | fam1 | fam1 | thrpt | 5 | 24423324 | ops/s | BenchmarkForNormal.new_compareBBKV | fam | fam1 | thrpt | 5 | 35369022 | ops/s | -3.00% BenchmarkForNormal.new_compareBBKV | fam1 | fam | thrpt | 5 | 35297691 | ops/s | -3.00% BenchmarkForNormal.new_compareKV | fam | fam1 | thrpt | 5 | 36468032 | ops/s | 1.00% BenchmarkForNormal.new_compareKV | fam1 | fam | thrpt | 5 | 36466929 | ops/s | 1.00% BenchmarkForNormal.new_compareKVVsBBKV | fam | fam1 | thrpt | 5 | 39476967 | ops/s | 0.00% BenchmarkForNormal.new_compareKVVsBBKV | fam1 | fam | thrpt | 5 | 38510761 | ops/s | -1.00% BenchmarkForNormal.old_compareBBKV | fam | fam1 | thrpt | 5 | 36428982 | ops/s | BenchmarkForNormal.old_compareBBKV | fam1 | fam | thrpt | 5 | 36412964 | ops/s | BenchmarkForNormal.old_compareKV | fam | fam1 | thrpt | 5 | 36102740 | ops/s | BenchmarkForNormal.old_compareKV | fam1 | fam | thrpt | 5 | 35996819 | ops/s | BenchmarkForNormal.old_compareKVVsBBKV | fam | fam1 | thrpt | 5 | 39376281 | ops/s | BenchmarkForNormal.old_compareKVVsBBKV | fam1 | fam | thrpt | 5 | 39022305 | ops/s | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519624496   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
Apache9 commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519609830 I applied the patch here and name the CellComparatorImpl as CellComparatorNewImpl, run the benchmark here https://github.com/Apache9/hbase-jmh-test/blob/master/src/main/java/org/apache/hadoop/hbase/ComparatorBenchmark.java The result is ``` Benchmark(p1) (p2) Mode Samples Score Score error Units o.a.h.j.ComparatorBenchmark.innerStoreCompareBBKVthrpt 20 55107486.443 1133989.704 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareBBKV fam1 thrpt 20 97610341.161 2164791.822 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareBBKVfam1thrpt 20 96814034.811 1846943.553 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareBBKVfam1 fam1 thrpt 20 54728226.906 763534.750 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKV thrpt 20 50255596.705 860022.699 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKVfam1 thrpt 20 105628757.244 1525111.142 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKV fam1thrpt 20 105053296.760 1424392.412 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKV fam1 fam1 thrpt 20 50580801.582 840448.402 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKVVsBBKVthrpt 20 56459332.399 1133748.467 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKVVsBBKV fam1 thrpt 20 99407694.992 2069495.268 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKVVsBBKVfam1thrpt 20 100109728.391 1683648.845 ops/s o.a.h.j.ComparatorBenchmark.innerStoreCompareKVVsBBKVfam1 fam1 thrpt 20 56310403.839 835453.447 ops/s o.a.h.j.ComparatorBenchmark.newCompareBBKV thrpt 20 49464136.602 1053482.976 ops/s o.a.h.j.ComparatorBenchmark.newCompareBBKV fam1 thrpt 20 83462821.561 1324986.585 ops/s o.a.h.j.ComparatorBenchmark.newCompareBBKV fam1thrpt 20 81859788.977 1456717.240 ops/s o.a.h.j.ComparatorBenchmark.newCompareBBKV fam1 fam1 thrpt 20 44728300.917 766115.081 ops/s o.a.h.j.ComparatorBenchmark.newCompareKV thrpt 20 48752157.933 756124.188 ops/s o.a.h.j.ComparatorBenchmark.newCompareKV fam1 thrpt 20 96669733.280 2070005.055 ops/s o.a.h.j.ComparatorBenchmark.newCompareKV fam1thrpt 20 87461522.072 1458003.674 ops/s o.a.h.j.ComparatorBenchmark.newCompareKV fam1 fam1 thrpt 20 41012509.734 812962.164 ops/s o.a.h.j.ComparatorBenchmark.newCompareKVVsBBKV thrpt 20 56058282.912 829861.076 ops/s o.a.h.j.ComparatorBenchmark.newCompareKVVsBBKV fam1 thrpt 20 96004134.969 1760285.343 ops/s o.a.h.j.ComparatorBenchmark.newCompareKVVsBBKV fam1thrpt 20 91558646.458 1706211.758 ops/s o.a.h.j.ComparatorBenchmark.newCompareKVVsBBKV fam1 fam1 thrpt 20 45120370.438 771026.541 ops/s o.a.h.j.ComparatorBenchmark.oldCompareBBKV thrpt 20 50129722.815 893317.939 ops/s o.a.h.j.ComparatorBenchmark.oldCompareBBKV fam1 thrpt 20 79861365.108 792186.386 ops/s o.a.h.j.ComparatorBenchmark.oldCompareBBKV fam1thrpt 20 83883234.894 1070778.517 ops/s o.a.h.j.ComparatorBenchmark.oldCompareBBKV fam1 fam1 thrpt 20 42874945.024 616270.961 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKV thrpt 20 46506918.063 863767.652 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKV fam1 thrpt 20 86728284.726 1606333.969 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKV fam1thrpt 20 95954923.114 1726842.910 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKV fam1 fam1 thrpt 20 42265300.903 855119.167 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKVVsBBKV thrpt 20 53619868.235 945846.574 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKVVsBBKV fam1 thrpt 20 91286001.006 1213908.825 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKVVsBBKV fam1thrpt 20 95828094.646 1037554.825 ops/s o.a.h.j.ComparatorBenchmark.oldCompareKVVsBBKV fam1 fam1 thrpt 20 47441790.062 697161.751 ops/s ``` Let me get the summary. -- This is an automated message from
[jira] [Commented] (HBASE-27109) Move replication queue storage from zookeeper to a separated HBase table
[
https://issues.apache.org/jira/browse/HBASE-27109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715623#comment-17715623
]
Hudson commented on HBASE-27109:
Results for branch HBASE-27109/table_based_rqs
[build #62 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/62/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/62/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/HBASE-27109%252Ftable_based_rqs/62/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move replication queue storage from zookeeper to a separated HBase table
>
>
> Key: HBASE-27109
> URL: https://issues.apache.org/jira/browse/HBASE-27109
> Project: HBase
> Issue Type: New Feature
> Components: Replication
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> This is a more specific issue based on the works which are already done in
> HBASE-15867.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27792) Guard Master/RS Dump Servlet behind admin walls
[ https://issues.apache.org/jira/browse/HBASE-27792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nihal Jain updated HBASE-27792: --- Status: Patch Available (was: Open) > Guard Master/RS Dump Servlet behind admin walls > --- > > Key: HBASE-27792 > URL: https://issues.apache.org/jira/browse/HBASE-27792 > Project: HBase > Issue Type: Improvement > Components: security, UI >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Minor > > Currently RSDumpServlet and MasterDumpServlet do not require any check for > whether the user has privileges to access to instrumentation servlets. > This is unlike other servlets like ProfileServlet, ConfServlet, > JMXJsonServlet etc. which are guarded by admin checks. > Goal of this JIRA is to add similar check for RS and Master Dump Servlet. > Post this change only admins will be able to access RSDumpServlet and > MasterDumpServlet, if _hadoop.security.instrumentation.requires.admin_ is > enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [hbase] bsglz commented on pull request #5171: HBASE-27788 Skip family comparing when compare cells inner the store
bsglz commented on PR #5171: URL: https://github.com/apache/hbase/pull/5171#issuecomment-1519449248 > Could you please run a benchmark on the current master code? We changed some methods from static to non static and also override some methods, I want to make sure that these modifications do not affect performance too much. > > Thanks, No problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
